October 31, 2022 to November 3, 2022
Clarion hotel Umeå
Europe/Amsterdam timezone

Review of HTC scheduling strategies for HEP at GridKa Tier 1

Nov 1, 2022, 3:20 PM
25m
Clarion hotel Umeå

Clarion hotel Umeå

Storgatan 36, Umeå, Sweden
Computing & Batch Services Computing and Batch Services

Speaker

Max Fischer (Karlsruhe Institute of Technology)

Description

Scheduling at large WLCG sites has to account for several peculiarities of the HEP usage profile: Prominently, the split into only 1-core and 8-core requests is known to lead to fragmentation. In addition, sites have to satisfy long-term and short-term fairshare, efficient job packing, internal flexibility and various other goals. Over the years, various strategies have been proposed in the community and implemented by sites at their own discretion.

We present a review of the strategies previously and currently used at the GridKa Tier 1 to tackle the HEP usage profile. We cover defragmentation, static versus dynamic partitioning, subgroups and more as well as their interplay. As a large grid site supporting several VOs and using the common HTCondor resource manager, we expect our experience to be applicable or at least educational for many sites.

Speaker release Yes
Presentation will be held... remotely

Primary author

Max Fischer (Karlsruhe Institute of Technology)

Co-authors

Andreas Petzold (KIT - Karlsruhe Institute of Technology (DE)) Matthias Schnepf

Presentation materials