Speaker
Description
The German CMS community (DCMS) as a whole can benefit from the various compute resources, available to its different institutes. While Grid-enabled and National Analysis Facility resources are usually shared within the community, local and recently enabled opportunistic resources like HPC centers and cloud resources are not. Furthermore, there is no shared submission infrastructure available.
In this contribution we present a concept, how connecting pools using HTCondors’s flocking and routing mechanisms allows for a transparent and more efficient usage of all resources available to the DCMS community. In addition to the statically provisioned resources, also dynamically allocated resources from external cloud providers can be integrated. However, the usage of such dynamically allocated resources gives rise to additional complexity. Constraints on access policies of the resources, as well as workflow necessities have to be taken care of.
To maintain a well-defined and reliable execution environment on each resource, virtualization and containerization technologies such as virtual machines, Docker, and Singularity, are used.
We give an overview about the concepts and first experiences on how to provide DCMS resources dynamically and transparently to the community.
