Speaker
Cedric Serfon
(CERN)
Description
The current ATLAS Distributed Data Management system (DQ2) is being replaced by a new one called Rucio. The new system has many improvements, but it requires a number of changes. One of the most significant ones is that no local file catalog like the LFC, which was a central component in DQ2, will be used by Rucio. Instead of querying a file catalogue that stores the association of files with their corresponding locations, in Rucio the physical path of a file can be ascertained by deriving it from the file's Logical File Name (LFN) via a deterministic function. Therefore, all file replicas produced by ATLAS have to be renamed before migrating to the new system. It represents about 300M files split between about 120 sites with six different storage technologies. An infrastructure to perform this is needed: it should be automated, robust, transparent for the users, fault tolerant, storage technology agnostic and require as little work as possible from site administrative personnel. It needs also to be fast enough to rename everything before the final switch to Rucio in 2014. An infrastructure following all these requirements has been developed and is described here. The technologies that have been used are also presented as well as the performance of the system.
Primary author
Cedric Serfon
(CERN)
Co-authors
Angelos Molfetas
(University of Sydney (AU))
Armin Nairz
(CERN)
Graeme Andrew Stewart
(CERN)
Dr
Luc Goossens
(CERN)
Mario Lassnig
(CERN)
Martin Barisits
(CERN)
Ralph Vigne
(University of Vienna (AT))
Thomas Beermann
(Bergische Universitaet Wuppertal (DE))
Vincent Garonne
(CERN)
