9-13 July 2018
Sofia, Bulgaria
Europe/Sofia timezone

HEPCloud, an Elastic Hybrid HEP Facility using an Intelligent Decision Support System

9 Jul 2018, 14:15
Hall 7 (National Palace of Culture)

Hall 7

National Palace of Culture

presentation Track 3 – Distributed computing T3 - Distributed computing


Eric Vaandering (Fermi National Accelerator Lab. (US))


HEPCloud is rapidly becoming the primary system for provisioning compute resources for all Fermilab-affiliated experiments. In order to reliably meet peak demands of the next generation of High Energy Physics experiments, Fermilab must either plan to locally provision enough resources to cover the forecasted need, or find ways to elastically expand its computational capabilities. Commercial cloud and allocation-based High Performance Computing (HPC) resources both have explicit and implicit costs that must be considered when deciding when to provision these resources, and to choose an appropriate scale. In order to support such provisioning in a manner consistent with organizational business rules and budget constraints, we have developed a modular intelligent decision support system (IDSS) to aid in the automatic provisioning of resources-spanning multiple cloud providers, multiple HPC centers, and grid computing federations. In this paper, we discuss the goals and architecture of the HEPCloud Facility, the architecture of the IDSS, and our early experience in using the IDSS for automated facility expansion both at Fermilab and Brookhaven Lab.

Primary authors

Alexander Moibenko (Fermi National Accelerator Laboratory) Anthony Tiradani (Fermilab) Burt Holzman (Fermi National Accelerator Lab. (US)) Dmitry Litvintsev (FNAL) Eileen Berman Eric Vaandering (Fermi National Accelerator Lab. (US)) Jim Kowalkowski (Fermilab) John Hover (Brookhaven National Laboratory (BNL)-Unknown-Unknown) Jose Caballero Bejar (Brookhaven National Laboratory (US)) Marc Paterno (Fermilab) Mine Altunay (Fermi National Accelerator Laboratory) Panagiotis Spentzouris (Fermilab) Parag Mhashilkar (Fermi National Accelerator Laboratory) Qiming Lu (Fermi National Accelerator Laboratory) Steven Timm (Fermi National Accelerator Lab. (US)) Dr Stuart Fuess (Fermilab) W.David Dagenhart (Fermi National Accelerator Lab. (US))

Presentation Materials