Speaker
Description
The ever growing amount of HEP data to be analyzed in the future requires as of today the allocation of additional, potentially only temporary available non-HEP dedicated resources. These so-called opportunistic resources are also well-suited to cover the typical unpredictable peak demands for computing resources in end-user analyses. However, their temporary availability requires a dynamic allocation, integration, and management, while their heterogeneity results in the challenge to allocate always the best matching resources in order to maintain a high resource utilization.
For this purpose, we are developing the multi-agent resource manager TARDIS (Transparent Adaptive Resource Dynamic Integration System) which reacts on the current utilization of the integrated resources. A feedback loop implemented by COBalD (Opportunistic Balancing Daemon) ensures the further allocation of well-used resources while reducing the amount of insufficiently used ones. TARDIS is able to allocate and manage resources from various resource providers such as HPC centers or commercial and public clouds while ensuring a dynamic allocation and efficient utilization of these heterogeneous opportunistic resources.
Furthermore, opportunistic resources are integrated into one overlay batch system which provides a single point of entry for all users. In order to provide the dedicated HEP software environment virtualization and container technologies are used.
In this contribution, we will give an overview of our developments of TARDIS/COBalD as well as the current status of the integration of opportunistic resources for HEP user analyses.
