25–29 Apr 2022
Europe/Zurich timezone

Rebalancing the HTCondor fairshare for mixed workloads

26 Apr 2022, 11:00
25m
Online workshop

Online workshop

Computing & Batch Services Computing & Batch Services

Speaker

Stefano Dal Pra (Universita e INFN, Bologna (IT))

Description

The INFN Tier-1 data centre is the main italian computing site for scientific communities on High Energy Physics and astroparticle research. Access to the resources is arbitrated by a HTCondor batch system which is in charge of balancing the overall usage by several competing user groups according to their agreed quotas. The different workloads submitted to the computing cluster is highly heterogeneous and a vast set of different requirements is to be considered by the batch system in order to provide user groups with a satisfactory fair share over the available resources. To prevent or reduce usage disparities a system to self adjust imbalances has been developed and it is being used with satisfactory results. This work explain how and when fair share implementations can miss optimal performances and describes a general method to improve them. Results of the current solution are presented and possible further developments are discussed.

Speaker release Yes

Author

Stefano Dal Pra (Universita e INFN, Bologna (IT))

Co-author

Dr Carmelo Pellegrino (INFN-CNAF)

Presentation materials