27–31 Mar 2023
Research Center for Environmental Changes (RCEC), Academia Sinica
Asia/Taipei timezone

Cloudscheduler V.2 from an HTCondor admin's point of view

30 Mar 2023, 14:55
25m
1F Conference Room (Research Center for Environmental Changes (RCEC), Academia Sinica )

1F Conference Room

Research Center for Environmental Changes (RCEC), Academia Sinica

128 Academia Road, Section 2 Nankang, Taipei 11529 Taiwan 25°2′45″N 121°36′37″E
Computing & Batch Services Computing and Batch Services

Speaker

Dr Marcus Ebert (University of Victoria)

Description

We developed a new version of Cloudscheduler on which we reported before from a technical point of view. Cloudscheduler is a system that manages VM resources on demand on local and remote compute clouds depending on job requirements and makes those VMs available to HTCondor pools. Via cloud-init and yaml files, VMs can be provisioned depending on the needs of a VO.
In this presentation, we will focus on our experience with the new Cloudscheduler running HTCondor jobs for HEP Grid jobs (Atlas and Belle-II), Astronomy (Dune) and HEP non-Grid jobs (BaBar) in a distributed HTCondor environment from a user and administrator point of view. We will show how it integrates with an existing HTCondor system and how it can be used to extend an existing pool with cloud resources when needed, for example in times of high demand or during down times of bare metal worker nodes, and how the system usage is monitored.

Author

Dr Marcus Ebert (University of Victoria)

Presentation materials