Usual welcome speech with some overview of DESY's activities
Some points worth emphasising
A brief introduction to HTCondor from a user perspective, covering how to submit and manage jobs and workflows.
An overview of how to administer an HTCondor pool.
In the context of the European Union(EU) Regulation on Genetically Modified Organisms (GMOs), validation of Polymerase Chain Reaction (PCR-)based detection methods is a fundamental task of the European Union Reference Laboratory for Geneticallly Modified Food and Feed (EU-RL GMFF) that requires to integrate the combination of both the experimental approach and of bioinformatics analyses for...
Learn how to configure job- and machine-specific policy.
This talk describes how IceCube uses HTCondor and an in-house glidein system to consolidate computing resources from a diverse set of facilities.
Flexible and Cost-effective Petabyte-Scale Architecture with HTCondor Processing and EOS Storage Backend for Earth Observation Applications
Veselin Vasilev, Dario Rodriguez, Armin Burger and Pierre Soille
European Commission, Joint Research Centre (JRC)
Directorate I. Competences. Unit I.3 Text and Data Mining
Via E. Fermi 2749, I-21027 Ispra (Va), Italy
The Copernicus...
Learn how HTCondor matches jobs to machines, and how to control it.
See how Azure and HTCondor are being used by TIFR for CMS.
After running for more than one year Grid resources in production on
HTCondor, we present our experiences on the HTCondor+ARCCE ecosystem.
Migrating existing resources from the previous batch system showed to be
easy to handle and is reassuring us about the scaling properties of
HTCondor.
In the last year, HTCondor has been enhanced with mechanisms to allow for kerberos tokens to be renewed. This is achieved in HTCondor with hooks for security credential producers, and security credential monitors. I propose to talk about using these methods, what we integrate with to handle the monitoring and renewal of kerberos tokens at CERN, operational experience in doing so, what we...
Local batch at DESY today is relying on afs $HOMEs heavily. For a transparent migration from SGE to HTCondor we want to make use of the Kerberos integration that is provided in terms of condor hooks in the current developer release of Condor. This talk will describe the tools we use to monitor and renew the kerberos tokens
in our condor environment.
This talk will detail the current status of the CERN migration from LSF to HTCondor. We will cover the grid and the HTCondor-CE, local submissions, monitoring/dashboards, feedback from users and the current progress.
Covers how to modify jobs during submission and deny unsuitable jobs.
A hands-on introduction to using HTCondor in Python.
An overview of recent developments and future plans in HTCondor.
Singularity linux containerization in HTCondor
A collection of little-known features added to HTCondor recently.
At the RAL Tier-1 we have been running HTCondor in production for 4 years. Currently the pool contains over 22000 cores. This talk will present an update on recent activity, including migration to the Docker universe, and future plans.
Purpose:
The objective of this project is to optimize Big Data (BD) workload scheduling, using a hybrid framework (dedicated and non-dedicated) that blends the best of both Hadoop YARN and HTCondor worlds in a single analytical environment.
Method:
The proposed OPERA-P, short for OPportunistically, Elastically Resource Allocation and Provisioning scheduler, is a new hybrid BD platform that...
The Open Science Grid is a partnership for a data-intensive research, focusing on technique distributed high-throughput computing (DHTC). At its core, the OSG often utilizes HTCondor to meet its distributed computing needs.
Within the OSG, HTCondor is used as a compute element, as a piece of the monitoring system, for pilot submission, for an information service -- and yes, for scheduling...
Advanced features in HTCondor for job submission.
A review of CMS experiment’s HTCondor Global Pool operations during the past year, including the challenges and solutions integrated into new HTCondor features which allowed us to reach scales of over 200K CPU cores world-wide.
GlideinWMS factory is a part of the CERN CMS global pool for submitting glideins to the grid sites, glideins work as placeholders, glideinWMS factory works on top of the HTCondor, when glideins reserve resources HTCondor scheduling jobs to grid sites.
How HTCondor deals with network architecture difficulties.
The HTCondor-CE provides a remote API on top of a local site batch system. The HTCondor-CE works best when paired with HTCondor, but it can provide a generic interface to other batch systems such as SLURM, PBS, or LSF. Originally developed within the Open Science Grid, the CE is now in use in sites around the world - from the smallest to largest of scales!
In this presentation, we will give...
Experiences with HTCondor Universes on a Petabyte Scale Platform for Earth Observation Data Processing
Dario Rodriguez, Veselin Vasilev, and Pierre Soille
European Commission, Joint Research Centre (JRC)
Directorate I. Competences. Unit I.3 Text and Data Mining
Via E. Fermi 2749, I-21027 Ispra (Va), Italy
Scientific and technical support to policies ranging from environment to...
This talk summarizes how the LIGO Scientific and Virgo collaborations
hunt for gravitational waves - tiny ripples in space-time created by
some of the most energetic events in the Universe. We will discuss
high-power lasers, extremely precise phase/distance measurements,
Einstein's equations, data analysis models and work-flows, largish
computing sites, Open Science Grid (OSG), and finally...
An overview of authentication and authorization in HTCondor.
A tool to easily expand an HTCondor pool via the Cloud.
A look at how the HTCondor pool at CHTC is transitioning from RHEL6 to RHEL7 with minimal user headaches.
A look at the scheduling policies used by CHTC at UW-Madison.
How to keep an eye on your HTCondor pool.
How to identify and fix things that frequently go wrong.
