9-13 July 2018
Sofia, Bulgaria
Europe/Sofia timezone

The ATLAS & Google "Data Ocean" Project for the HL-LHC era

9 Jul 2018, 15:45
Hall 8 (National Palace of Culture)

Hall 8

National Palace of Culture

presentation Track 4 - Data Handling T4 - Data handling


Mario Lassnig (CERN)


With the LHC High Luminosity upgrade the workload and data management systems are facing new major challenges. To address those challenges ATLAS and Google agreed to cooperate on a project to connect Google Cloud Storage and Compute Engine to the ATLAS computing environment. The idea is to allow ATLAS to explore the use of different computing models, to allow ATLAS user analysis to benefit from the Google infrastructure, and to give Google real science use cases to improve their cloud platform. Making the output of a distributed analysis from the grid quickly available to the analyst is a difficult problem. Redirecting the analysis output to Google Cloud Storage can provide an alternative, faster solution for the analyst. First, Google's Cloud Storage will be connected to the ATLAS Data Management System Rucio. The second part aims to let jobs run on Google Compute Engine, accessing data from either ATLAS storage or Google Cloud Storage. The third part involves Google implementing a global redirection between their regions to expose Google Cloud Storage as a single global entity. The last part will deal with the economic model necessary for sustainable cloud resource usage, including Google Cloud Storage costs, network costs, and peering costs with ESnet.

Primary authors

Mario Lassnig (CERN) Alexei Klimentov (Brookhaven National Laboratory (US)) Kaushik De (University of Texas at Arlington (US)) Karan Bhatia Fernando Harald Barreiro Megino (University of Texas at Arlington) Martin Barisits (CERN) Cedric Serfon (University of Oslo (NO)) Thomas Beermann (CERN) Ruslan Mashinistov (Russian Academy of Sciences (RU)) Torre Wenaus (Brookhaven National Laboratory (US)) Tobias Wegner (Bergische Universitaet Wuppertal (DE)) Vincent Garonne (University of Oslo (NO))

Presentation Materials