Speaker
Mr
Jan Veverka
(Massachusetts Inst. of Technology (US))
Description
The data acquisition system (DAQ) of the CMS experiment at the CERN Large Hadron Collider assembles events at a rate of 100 kHz, transporting event data at an aggregate throughput of 100 GByte/s to the high level trigger (HLT) farm. The HLT farm selects interesting events for storage and offline analysis at a rate of around 1 kHz. The DAQ system has been redesigned during the accelerator shutdown in 2013/14. The motivation is twofold: Firstly, the current compute nodes, networking, and storage infrastructure will have reached the end of their lifetime by the time the LHC restarts. Secondly, in order to handle higher LHC luminosities and event pileup, a number of sub-detectors will be upgraded, increasing the number of readout channels and replacing the off-detector readout electronics with a μTCA implementation. The new architecture will take advantage of the latest developments in the computing industry. For data concentration, 10/40 Gbit Ethernet technologies will be used, as well as an implementation of a reduced TCP/IP in FPGA for a reliable transport between custom electronics and commercial computing hardware. A 56 Gbps Infiniband FDR CLOS network has been chosen for the event builder with a throughput of ~4 Tbps. The HLT processing is entirely file based. This allows the DAQ and HLT systems to be independent, and to use the same framework for the HLT as for the offline processing. The fully built events are sent to the HLT with 1/10/40 Gbit Ethernet via network file systems. Hierarchical collection of HLT accepted events and monitoring meta-data are stored into a global file system. This paper presents the requirements, technical choices, and performance of the new system.
Primary author
Frans Meijers
(CERN)
Co-authors
Andre Georg Holzner
(Univ. of California San Diego (US))
Andrea Petrucci
(CERN)
Andrew Kevin Forrest
(University of Kent (GB))
Dr
Attila Racz
(CERN)
Aymeric Arnaud Dupont
(CERN)
Benjamin Stieger
(CERN)
Carlos Nunez Barranco Fernandez
(CERN)
Christian Deldicque
(CERN)
Christoph Paus
(Massachusetts Inst. of Technology (US))
Christoph Schwick
(CERN)
Dominique Gigi
(CERN)
Emilio Meschi
(CERN)
Frank Glege
(CERN)
Georgiana Lavinia Darlea
(Massachusetts Inst. of Technology (US))
Guillelmo Gomez Ceballos Retuerto
(Massachusetts Inst. of Technology (US))
Hannes Sakulin
(CERN)
James Gordon Branson
(Univ. of California San Diego (US))
Mr
Jan Veverka
(Massachusetts Inst. of Technology (US))
Dr
Jeroen Hegeman
(CERN)
Konstanty Sumorok
(Massachusetts Inst. of Technology (US))
Lorenzo Masetti
(CERN)
Luciano Orsini
(CERN)
Dr
Marc Dobson
(CERN)
Marco Pieri
(Univ. of California San Diego (US))
Olivier Chaze
(CERN)
Petr Zejdl
(CERN)
Remi Mommsen
(Fermi National Accelerator Lab. (US))
Robert Gomez-Reino Garrido
(CERN)
Samim Erhan
(Univ. of California Los Angeles (US))
Sergio Cittolin
(Univ. of California San Diego (US))
Srecko Morovic
(CERN)
Tomasz Adrian Bawej
(Warsaw University of Technology (PL))
Ulf Behrens
(Deutsches Elektronen-Synchrotron (DE))
Vivian O'Dell
(Fermi National Accelerator Lab. (US))
