Speaker
Georgiana Lavinia Darlea
(Massachusetts Inst. of Technology (US))
Description
The CMS experiment at CERN is one of the two general-purpose detectors on the Large Hadron Collider (LHC) in the Geneva area, Switzerland. Its infrastructure has undergone massive upgrades during 2013 and 2014, which lead to major changes in the philosophy of its DAQ (Data AcQuisition) system. One of the major components of this system is the Storage Manager, which is responsible for buffering the online data generated at the level of the readout units (RUs), which crosses the computing farm undergoing different processing and filtering stages all the way to the builder units (BUs). The Storage Manager at CMS is made up of three components: the distributed file system, the Merger service and the Transfer System. In the production DAQ system there will be around 50 BUs which will concurrently write their output data at an expected aggregated rate of 2 GB/s. A merger service has been put in place to aggregate this data. Counting the merger reading of the data provided by the BUs, its writing back of the merged data and the final reading for transfer to Tier0, an estimated bandwidth of 7GB/s in concurrent r/w mode is needed. Moreover, the Storage Manager has to be able to cope with being disconnected from Tier0 for 1 week, so an estimated of 250TB of total usable disk space is required. A unified name space file system (Lustre) has been chosen in order to cope with these requirements. Three different implementations of the merger service are proposed, each one providing different advantages: robustness, easy debugging, bandwidth requirements reduction. The merger is also providing the bookkeeping needed for establishing the handshake protocol between the Transfer System and the Tier0 facility at CERN, which ensures data consistency and integrity throughout the transfer process. Even if the handshake protocol itself is identical to the one from the Run1, some adjustments needed to be performed in the Transfer System in order to accommodate the new structures providing the required meta-information related to the acquired data. In addition to the nominal transfer of data to Tier0, the Transfer System needs to intelligently distribute the data, for a number of data streams need to be (also) stored locally in the CMS network for various consumers to process on site. In this article we present the various technological and implementation choices of the three components of the Storage Manager.
Primary authors
Anastasios Andronidis
(Aristotle Univ. of Thessaloniki (GR))
Andre Georg Holzner
Andrea Petrucci
(CERN)
Andrew Kevin Forrest
(University of Kent (GB))
Dr
Attila Racz
(CERN)
Aymeric Arnaud Dupont
(CERN)
Benjamin Stieger
(CERN)
Carlos Nunez Barranco Fernandez
(CERN)
Christian Deldicque
(CERN)
Christoph Paus
(Massachusetts Inst. of Technology (US))
Christoph Schwick
(CERN)
Christopher Colin Wakefield
(Staffordshire University (GB))
Dominique Gigi
(CERN)
Emilio Meschi
(CERN)
Frank Glege
(CERN)
Frans Meijers
(C)
Georgiana Lavinia Darlea
(Massachusetts Inst. of Technology (US))
Guillelmo Gomez Ceballos Retuerto
(Massachusetts Inst. of Technology (US))
Hannes Sakulin
(CERN)
James Gordon Branson
(Univ. of California San Diego (US))
Mr
Jan Veverka
(Massachusetts Inst. of Technology (US))
Jean-Marc Olivier Andre
(Fermi National Accelerator Lab. (US))
Dr
Jeroen Hegeman
(CERN)
Konstanty Sumorok
(Massachusetts Inst. of Technology (US))
Lorenzo Masetti
(CERN)
Luciano Orsini
(CERN)
Dr
Marc Dobson
(CERN)
Marco Pieri
(Univ. of California San Diego (US))
Olivier Chaze
(CERN)
Petr Zejdl
(CERN)
Remi Mommsen
(Fermi National Accelerator Lab. (US))
Samim Erhan
(Univ. of California Los Angeles (US))
Sergio Cittolin
(Univ. of California San Diego (US))
Srecko Morovic
(CERN)
Tomasz Adrian Bawej
(University of Wisconsin (US))
Ulf Behrens
(Deutsches Elektronen-Synchrotron (DE))
Vivian O'Dell
(Fermi National Accelerator Laboratory (FNAL))
