Jul 9 – 13, 2018
Sofia, Bulgaria
Europe/Sofia timezone

Exploring GlideinWMS and HTCondor scalability frontiers for an expanding CMS Global Pool

Jul 12, 2018, 12:15 PM
15m
Hall 7 (National Palace of Culture)

Hall 7

National Palace of Culture

presentation Track 3 – Distributed computing T3 - Distributed computing

Speaker

Antonio Perez-Calero Yzquierdo (Centro de Investigaciones Energéti cas Medioambientales y Tecno)

Description

The CMS Submission Infrastructure Global Pool, built on GlideinWMS and HTCondor, is a worldwide distributed dynamic pool responsible for the allocation of resources for all CMS computing workloads. Matching the continuously increasing demand for computing resources by CMS requires the anticipated assessment of its scalability limitations. Extrapolating historical usage trends, by LHC Run III the CMS Global Pool must be able to manage stably and efficiently 0.5M CPU cores, about a factor 2 from current size. In addition, the Global Pool must be able to expand in a more heterogeneous environment, in terms of resource provisioning (combining Grid, HPC and Cloud) and workload submission. A dedicated testbed has been set up to simulate such conditions with the purpose of finding potential bottlenecks in the software or its configuration. This contribution will provide a thorough description of the various scalability dimensions in size and complexity that are being explored for the future Global Pool, along with the analysis and solutions to the limitations proposed with the support of the GlideinWMS and HTCondor developer teams.

Primary authors

Antonio Perez-Calero Yzquierdo (Centro de Investigaciones Energéti cas Medioambientales y Tecno) James Letts (Univ. of California San Diego (US)) David Alexander Mason (Fermi National Accelerator Lab. (US)) Diego Davila Foyo (Autonomous University of Puebla (MX)) Brian Paul Bockelman (University of Nebraska Lincoln (US)) Farrukh Aftab Khan (Fermi National Accelerator Lab. (US)) Amjad Kotobi (University of Malaya (MY)) Krista Larson (Fermi National Accelerator Lab. (US)) Kenyi Paolo Hurtado Anampa (University of Notre Dame (US)) Todor Trendafilov Ivanov (University of Sofia (BG)) Marco Mascheroni (Univ. of California San Diego (US))

Presentation materials