Oct 10 – 14, 2016
San Francisco Marriott Marquis
America/Los_Angeles timezone

Experiences with the new ATLAS Distributed Data Management System

Oct 11, 2016, 2:45 PM
15m
GG C3 (San Francisco Mariott Marquis)

GG C3

San Francisco Mariott Marquis

Oral Track 4: Data Handling Track 4: Data Handling

Speaker

Vincent Garonne (University of Oslo (NO))

Description

The ATLAS Distributed Data Management (DDM) system has evolved drastically in the last two years with the Rucio software fully
replacing the previous system before the start of LHC Run-2. The ATLAS DDM system manages now more than 200 petabytes spread on 130
storage sites and can handle file transfer rates of up to 30Hz. In this talk, we discuss our experience acquired in developing,
commissioning, running and maintaining such a large system. First, we describe the general architecture of the system, our
integration with external services like the WLCG File Transfer Service and the evolution of the system over its first year of
production. Then, we show the performance of the system, describe the integration of new technologies such as object stores, and
outline future developments which mainly focus on performance and automation. Finally we discuss the long term evolution of ATLAS
data management.

Primary Keyword (Mandatory) Distributed data handling
Secondary Keyword (Optional) Storage systems

Primary author

Vincent Garonne (University of Oslo (NO))

Presentation materials