10–14 Oct 2016
San Francisco Marriott Marquis
America/Los_Angeles timezone

The CMS Data Acquisition - Architectures for the Phase-2 Upgrade

11 Oct 2016, 11:00
15m
Sierra A (San Francisco Mariott Marquis)

Sierra A

San Francisco Mariott Marquis

Oral Track 1: Online Computing Track 1: Online Computing

Speaker

Emilio Meschi (CERN)

Description

In Long Shutdown 3 the CMS Detector will undergo a major upgrade to prepare for the second phase of the LHC physics program, starting around 2026. The HL-LHC upgrade will bring instantaneous luminosity up to 5x10^34 cm-2 s-1 (levelled), at the price of extreme pileup of 200 interactions per crossing. A new silicon tracker with trigger capabilities and extended coverage, and new high granularity forward calorimetry will enhance the CMS acceptance and selection power. This will enable precision measurements of the Higgs boson properties, as well as extend the discovery reach for physics beyond the standard model, while coping with conditions dictated by the HL-LHC parameters.
Following the tradition, the CMS Data Acquisition System will continue to feature two trigger levels.
The detector will be read out at an unprecedented data rate of up to 50 Tb/s read at a Level-1 rate of 750 kHz from some 50k high-speed optical detector links, for an average expected event size of 5MB. Complete events will be analysed by a software trigger (HLT) running on standard processing nodes, and selected events will be stored permanently at a rate of up to 10 kHz for offline processing and analysis.

In this paper we discuss the baseline design of the DAQ and HLT systems for the Run 4, taking into account the projected evolution of high speed network fabrics for event building and distribution, and the anticipated performance of many-core CPU and their memory and I/O architectures. Assuming a modest improvement of the processing power of 12.5% per year for the standard Intel architecture CPU and the affordability, by 2026, of 100-200 Gb/s links, and scaling the current HLT CPU needs for increased event size, pileup, and rate, the CMS DAQ will require about:

  • 800 100 Gb/s links connecting 400 servers for event building
  • 4500 processing servers, or a total of about 11 MHS for HLT

Implications on hardware and infrastructure requirements for the DAQ “data center” are analysed. Emerging technologies for data reduction, in particular of CPU-FPGA hybrid systems, but also alternative CPU architectures, are considered. These technologies may in the future help containing the TCO of the system, while improving the energy performance and reducing the cooling requirements.
Novel possible approaches to event building and online processing are also examined, which are inspired by trending developments in other areas of computing dealing with large masses of data.

We conclude by discussing the opportunities offered by reading out and processing parts of the detector, wherever the front-end electronics allows, at the machine clock rate (40 MHz). While the full detector is being read out and processed at the Level-1 rate, a second, parallel DAQ system would run as an "opportunistic experiment” processing tracker trigger and calorimeter data at 40 MHz. This idea presents interesting challenges and its physics potential should be studied.

Primary Keyword (Mandatory) DAQ
Secondary Keyword (Optional) Trigger
Tertiary Keyword (Optional) High performance computing

Primary author

Co-authors

Andre Georg Holzner (Univ. of California San Diego (US)) Attila Racz (CERN) Ben Craigs (CERN, Geneva, Switzerland) Christian Deldicque (CERN) Christoph Paus (Massachusetts Inst. of Technology (US)) Christoph Schwick (CERN) Cristian Contescu (Fermi National Accelerator Lab. (US)) Dainius Simelevicius (CERN) Dominique Gigi (CERN) Frank Glege (CERN) Frans Meijers (CERN) Georgiana Lavinia Darlea (Massachusetts Inst. of Technology (US)) Guillelmo Gomez Ceballos Retuerto (Massachusetts Inst. of Technology (US)) Hannes Sakulin (CERN) James Gordon Branson (Univ. of California San Diego (US)) Jean-Marc Olivier Andre (Fermi National Accelerator Lab. (US)) Jeroen Hegeman (CERN) Jonathan Fulcher (CERN) Lorenzo Masetti (CERN) Luciano Orsini (CERN) Marc Dobson (CERN) Marco Pieri (Univ. of California San Diego (US)) Nicolas Doualot (Fermi National Accelerator Lab. (US)) Olivier Chaze (CERN) Petr Zejdl (Fermi National Accelerator Lab. (US)) Philipp Maximilian Brummer (CERN, Geneva, Switzerland) Raul Jimenez Estupinan (CERN) Remi Mommsen (Fermi National Accelerator Lab. (US)) Samim Erhan (Univ. of California Los Angeles (US)) Sergio Cittolin (Univ. of California San Diego (US)) Srecko Morovic (CERN) Thomas Reis (CERN) Ulf Behrens (Deutsches Elektronen-Synchrotron (DE)) Vivian O'Dell (Fermi National Accelerator Laboratory (FNAL)) Zeynep Demiragli (Massachusetts Inst. of Technology (US))

Presentation materials