Jul 9 – 13, 2018
Sofia, Bulgaria
Europe/Sofia timezone

Using PanDA WMS for LSST Simulations on Distributed Infrastructure

Jul 12, 2018, 11:30 AM
Hall 7 (National Palace of Culture)

Hall 7

National Palace of Culture

presentation Track 3 – Distributed computing T3 - Distributed computing


Pavlo Svirin (Brookhaven National Laboratory (US))


A goal of LSST (Large Synoptic Survey Telescope) project is to conduct a 10-year survey of the sky that is expected to deliver 200 petabytes of data after it begins full science operations in 2022. The project will address some of the most pressing questions about the structure and evolution of the universe and the objects in it. It will require a large amount of simulations to understand the collected data.
In this talk we will discuss the use of the PanDA Workload Management System (WMS) for LSST photon simulations, which model the atmosphere, optics and camera. The PanDA WMS was developed by the ATLAS Experiment at the LHC to manage data analysis and detector simulations on distributed and heterogeneous computing resources. It now runs ~2M jobs per day on hundreds of Grid sites and serving thousands of ATLAS users. PanDA processed about 1.5 exabytes of data in 2017. PanDA is also used for projects and experiments outside ATLAS and is expected to be suitable for computational needs of LSST Dark Energy Science Collaboration (DESC).
For running LSST simulations with the PanDA WMS we have established a distributed testbed infrastructure that employs the resources of several sites on GridPP and Open Science Grid as well as the Titan supercomputer at ORNL. In order to submit jobs to these sites we have used a PanDA server instance deployed on the Amazon AWS Cloud. We will present results of running realistic LSST simulations with this setup and discuss future plans for integrations of PanDA with LSST computational infrastructure.

Primary authors

Pavlo Svirin (Brookhaven National Laboratory (US)) Sergey Panitkin (Brookhaven National Laboratory (US)) Danila Oleynik (Joint Institute for Nuclear Research (RU)) Ruslan Mashinistov (Russian Academy of Sciences (RU)) Alessandra Forti (University of Manchester (GB)) Peter Love (Lancaster University (GB)) Mr Anže Slosar (Brookhaven National Laboratory) Andrei Nomerotski (Brookhaven National Laboratory) Mr Erin Sheldon (Brookhaven National Laboratory) Ms Hyeyun Park (Stony Brook University) Alexei Klimentov (Brookhaven National Laboratory (US)) Torre Wenaus (Brookhaven National Laboratory (US))

Presentation materials