We upgraded Indico to version 3.0. The new search is now available as well.
10-15 March 2019
Steinmatte conference center
Europe/Zurich timezone

HEP Analyses on Dynamically Allocated Opportunistic Computing Resources

Not scheduled
20m
Steinmatte conference center

Steinmatte conference center

Hotel Allalin, Saas Fee, Switzerland https://allalin.ch/conference/
Poster Track 1: Computing Technology for Physics Research Poster Session

Speaker

Matthias Jochen Schnepf (KIT - Karlsruhe Institute of Technology (DE))

Description

The ever growing amount of HEP data to be analyzed in the future requires as of today the allocation of additional, potentially only temporary available non-HEP dedicated resources. These so-called opportunistic resources are also well-suited to cover the typical unpredictable peak demands for computing resources in end-user analyses. However, their temporary availability requires a dynamic allocation, integration, and management, while their heterogeneity results in the challenge to allocate always the best matching resources in order to maintain a high resource utilization.

For this purpose, we are developing the multi-agent resource manager TARDIS (Transparent Adaptive Resource Dynamic Integration System) which reacts on the current utilization of the integrated resources. A feedback loop implemented by COBalD (Opportunistic Balancing Daemon) ensures the further allocation of well-used resources while reducing the amount of insufficiently used ones. TARDIS is able to allocate and manage resources from various resource providers such as HPC centers or commercial and public clouds while ensuring a dynamic allocation and efficient utilization of these heterogeneous opportunistic resources.

Furthermore, opportunistic resources are integrated into one overlay batch system which provides a single point of entry for all users. In order to provide the dedicated HEP software environment virtualization and container technologies are used.

In this contribution, we will give an overview of our developments of TARDIS/COBalD as well as the current status of the integration of opportunistic resources for HEP user analyses.

Primary author

Matthias Jochen Schnepf (KIT - Karlsruhe Institute of Technology (DE))

Co-authors

Ralf Florian Von Cube (Rheinisch-Westfaelische Tech. Hoch. (DE)) Max Fischer (GSI - Helmholtzzentrum fur Schwerionenforschung GmbH (DE)) Manuel Giffels (KIT - Karlsruhe Institute of Technology (DE)) Christoph Heidecker (KIT - Karlsruhe Institute of Technology (DE)) Andreas Heiss (KIT - Karlsruhe Institute of Technology (DE)) Eileen Kuehn (KIT - Karlsruhe Institute of Technology (DE)) Andreas Petzold (KIT - Karlsruhe Institute of Technology (DE)) Gunter Quast (KIT - Karlsruhe Institute of Technology (DE)) Martin Benedikt Sauter (KIT - Karlsruhe Institute of Technology (DE))

Presentation materials

Peer reviewing

Paper