Provides an overview of HTCondor's design and the principles behind it
An in-depth coverage of the HTCondor classad language with examples.
An overview of HTCondor's official Python APIs, including job submission, monitoring, and what's new. Beyond just pointpoint files, this talk will attempt to allow people to learn live via Jupyter notebook sessions.
HTCondor Advanced Job Submission
An explanation of the algorithms and policies of the HTCondor negotiator
An HTC cluster using HTCondor has been set up at Bonn University in 2017/2018.
All infrastructure is fully puppetised, including the HTCondor configuration.
Both interactive and batch jobs are run inside Singularity containers, and users only have to choose the desired OS via a job parameter from an offered collection of container images without setting up or building a container themselves....
In this talk we provide new details of the DESY configurations for HTCondor. We focus on features needed for user registry integration, node maintenance operations and fair share / quota handling. We are working on Docker, Jupyter and GPU integration into our smooth and transparent operating model setup.
Introduction to some recent work by the HTCondor team to enable Python code, including Python embedded in Jupyter Notebooks, to easily and naturally leverage high throughput computing via HTCondor.
In the framework of the Common Agricultural Policy (CAP) of the European Union, a big technological shift is happening. For decades, the correct payment of subsidies to farmers was controlled by means of remotely sensed images, by doing visual interpretation and field visits, to assess that a randomly selected percentage of the land parcels respected all the rules. In recent years, we are...
Overview of new features recently released as well as discussion about the HTCondor development roadmap.
Replay of second part of DESY presentation of Tue 24 Sep
The CERN HTCondor pool is currently offering 200K cores of compute power to hundreds of users in the HEP community. Managing such cluster requires a significant effort in the daily operations, not only because of the scale, but also because of the diversity of the resources. In this scenario, the adoption of automation and monitoring tools becomes a strong requirement to optimize both the...
The overwhelming majority of batch workload at CERN is very similar. Experiment pilots, experiment tier-0 production, and local users running single core jobs. However one size doesn't fit all, and we now have a number of different edge cases that this talk will cover...
GPUs, from machine learning to software validation.
Users who are in a grey area between HPC & HTC
Lower priority...
Describes the several ways more than one condor pool can be joined.
Discussion about how HTCondor jobs can access their data, including some recent developments.
Discusses work in program about possible RESTful interfaces to HTCondor
Group-owned and operated clusters at the Physics Department of the University of Milan (UNIMI) are pooled together via the HTCondoe 'Startd flocking' feature. We describe the setup, its applications and possible use for parallel applications, with some preliminary performance results.
The resource needs of high energy physics experiments such as CMS at the LHC are expected to continue to grow significantly over the next decade, and will be more and more satisfied by computing capacity with non-standard characteristics. This presents challenges not only of scale but of complexity in resource provisioning and allocation. In this contribution, we will present results of recent...
One of the main challenges the JRC Big Data Platform-JEODPP [1] is to offer well consolidated computational services, such as the batch system or interactive data visualization on which users can process large scale geospatial data while ensuring a smooth user experience combined with easy administration of all resources from hardware to applications.
Due to the heterogeneity of the user...
In our Directorate F Health, Consumers and Reference Materials at JRC, the Knowledge for Health and Consumer Safety Unit F.7 deals with anticipating knowledge needs, mapping knowledge gaps and suggesting research topics to be carried out in the Directorate and possibly in the JRC. For example, thousands of publications are released every year on different topics where JRC has strong competence...
Overview of the design of the HTCondor CE
An introduction to HTCondor-CE including an overview of its architecture and supported batch systems.
An overview of HTCondor-CE configuration with an emphasis on Job Router semantics
At CC-IN2P3, CREAM-CEs are being used since grid was deployed. Now, that we are evaluating to move our grid production from UGE to HTCondor a logical step was to evaluate also HTCondor-CEs.
In this presentation we show our experience installing, configuring and monitoring the cluster, some results and also our future plans and requirements.
An overview of the current support for APEL (accounting) and BDII (information services)
An overview of common issues encountered when supporting an HTCondor-CE, where to find additional information, and how to resolve them
Upcoming features and direction of HTCondor-CE
Reflections on the common upcoming requirements and directions of the scientific high throughput computing community.
With more than 2,000 bioinformatics tools available, Usegalaxy.eu (https://usegalay.eu) is the biggest Galaxy instance in Europe covering most of the hottest bioinformatics topics and communities.
After one year from its public launch into March 2018, Usegalaxy.eu has reached the important milestone of 5 million jobs executed and over 6 thousand registered users.
Several computer centers...
\documentclass{article}
\usepackage{filecontents}
\usepackage{authblk}
\usepackage{natbib}
\usepackage{natbib}
\bibliographystyle{abbrvnat}
\setcitestyle{numbers,open={[},close={]},citesep={,}}
\begin{filecontents}{\jobname.bib}
@article{phillips2008a,
author = {Phillips, S.J. and Dudík, M.},
title = {Modeling of species distributions with Maxent: new extensions and a...
The spatial distribution of built-up areas and their expansion represent one of the most important forms of land use/ land cover changes confronting climate, environmental and socio-economic systems at a global scale. Characterizing the status and dynamics of built-up areas over large areas is technically feasible thanks to the availability of a panoply of earth observation data with different...
Overview of the security mechanisms available in HTCondor, including discussion on security configuration and new security features being introduced in the HTCondor v8.9 series.
Presentation on work to integrate distributed authorization technologies such as SciTokens and OAuth 2.0 into HTCondor, and what this means for end-users and system administrators.
Container support in HTCondor
Overview of HTCondor's architecture and sequence of events when scheduling jobs.
