Opportunistic transparent extension of a WLCG Tier 2 center using HPC resources

20 May 2021, 11:42
13m
Short Talk Distributed Computing, Data Management and Facilities Virtualisation

Speaker

Ralf Florian Von Cube (KIT - Karlsruhe Institute of Technology (DE))

Description

Computing resource needs are expected to increase drastically in the future. The HEP experiments ATLAS and CMS foresee an increase of a factor of 5-10 in the volume of recorded data in the upcoming years. The current infrastructure, namely the WLCG, is not sufficient to meet the demands in terms of computing and storage resources.

The usage of non HEP specific resources is one way to reduce this shortage. However, using them comes at a cost: First, with multiple of such resources at hand, it gets more and more difficult for the single user, as each resource normally requires its own authentication and has its own way of accessing it. Second, as they are not specifically designed for HEP workflows, they might lack dedicated software or other necessary services.

Allocating the resources at the different providers can be done by COBalD/TARDIS, developed at KIT. The resource manager integrates resources on demand into one overlay batch system, providing the user with a single point of entry. The software and services, needed for the communities workflows, are transparently served through containers.

With this, an HPC cluster at RWTH Aachen University is dynamically and transparently integrated into a tier 2 WLCG resource, virtually doubling its computing capacities.

Primary authors

Ralf Florian Von Cube (KIT - Karlsruhe Institute of Technology (DE)) Rene Caspart (KIT - Karlsruhe Institute of Technology (DE)) Max Fischer (Karlsruhe Institute of Technology) Manuel Giffels (KIT - Karlsruhe Institute of Technology (DE)) Eileen Kuehn (Karlsruhe Institute of Technology) Gunter Quast (KIT - Karlsruhe Institute of Technology (DE)) Matthias Jochen Schnepf (KIT - Karlsruhe Institute of Technology (DE))

Presentation materials

Proceedings

Paper