Speaker
Parag Mhashilkar
(Fermi National Accelerator Laboratory)
Description
As part of the Fermilab/KISTI cooperative research project, Fermilab has successfully run an experimental simulation workflow at scale on a federation of Amazon Web Services (AWS), FermiCloud, and local FermiGrid resources. We used the CernVM-FS (CVMFS) file system to deliver the application software. We established Squid caching servers in AWS as well, using the Shoal system to let each individual virtual machine find the closest squid server. We also developed an automatic virtual machine conversion system so that we could transition virtual machines made on FermiCloud to Amazon Web Services.
We used this system to successfully run a cosmic ray simulation of the NOvA detector at Fermilab, making use of both AWS spot pricing and network bandwidth discounts to minimize the cost. On FermiCloud we also were able to run the workflow at the scale of 1000 virtual machines, using a private network routable inside of Fermilab. We present the details of the technological improvements that were used to make this successfully.
Primary authors
Dr
Gabriele Garzoglio
(FERMI NATIONAL ACCELERATOR LABORATORY)
Steven Timm
(Fermilab)
Co-authors
Gerard Bernabeu
(FERMI NATIONAL ACCELERATOR LABORATORY)
Dr
Hyunwoo Kim
(FERMI NATIONAL ACCELERATOR LABORATORY)
Joe Boyd
(FERMI NATIONAL ACCELERATOR LABORATORY)
Neha Sharma
(FERMI NATIONAL ACCELERATOR LABORATORY)
Nicholas Peregonow
(FERMI NATIONAL ACCELERATOR LABORATORY)
Parag Mhashilkar
(Fermi National Accelerator Laboratory)
SANDEEP PALUR
(ILLINOIS INSTITUTE OF TECHNOLOGY)
Seoyoung Noh
(KISTI Korea Institute of Science & Technology Information (KR))
