Speaker
Description
DESY manages not only one of the largest Tier-2 sites with about 18 500 CPU cores for Grid workloads but also about 8000 CPU cores for interactive user analyses. In this presentation, we recapitulate the consolidation of the batch systems in a common HTCondor based setup and the lessons learned as both use cases differ in their goals. Followingly, we will give an outlook
on the future developments.\newline
While for Grid jobs startup latencies are negligible and the primary focus is on an optimal utilization of the resources, users of the {\it National Analysis Factory} for interactive analyses prefer a high responsiveness of the batch system as well as the storage.\newline
In the ongoing evolution of the batch system, we are exploring two different approaches to abstract the batch node's host OS from the actual job OS. For Grid jobs we are running legacy workloads in lightweight Singularity containers deployed via CVMFS. For interactive NAF jobs, we move towards Docker containers for a more heavy weight replication of a full batch node to provide the users with their full set of accustomed tools.\newline
To utilize resources best we further investigate the opportunistically backfilling resources, especially GPU nodes, between pools without interfering with user experiences.
| Consider for promotion | No |
|---|
