Indico has been updated to v3.3. See our blog post for details on this release. (OTG0146394)

File-based data flow in the CMS Filter Farm

Apr 14, 2015, 3:15 PM
Village Center (Village Center)

Village Center

Village Center

oral presentation Track1: Online computing Track 1 Session


Emilio Meschi (CERN)


During the LHC Long Shutdown 1, the CMS DAQ system underwent a partial redesign to replace obsolete network equipment, use more homogeneous switching technologies, and prepare the ground for future upgrades of the detector front-ends. The software and hardware infrastructure to provide input, execute the High Level Trigger (HLT) algorithms and deal with output data transport and storage has also been redesigned to be completely file-based. This approach provides a complete decoupling between the HLT algorithms and the input and output data flow. All the metadata needed for bookkeeping of the data flow and the HLT process lifetimes are also generated in the form of small “documents” using the JSON encoding, by either services in the flow of the HLT execution (for rates etc.) or watchdog processes. These “files" can remain memory-resident or be written to disk if they are to be used in another part of the system (e.g. for aggregation of output data). We discuss how this redesign improves to the robustness and flexibility of the CMS DAQ and the performance of the system currently being commissioned for the LHC Run 2.

Primary author


Anastasios Andronidis (Aristotle Univ. of Thessaloniki (GR)) Andre Georg Holzner (Univ. of California San Diego (US)) Andrea Petrucci (CERN) Dr Attila Racz (CERN) Aymeric Arnaud Dupont (CERN) Benjamin Stieger (CERN) Carlos Nunez Barranco Fernandez (CERN) Christian Deldicque (CERN) Christoph Paus (Massachusetts Inst. of Technology (US)) Christoph Schwick (CERN) Dominique Gigi (CERN) Frank Glege (CERN) Frans Meijers (CERN) Georgiana Lavinia Darlea (Massachusetts Inst. of Technology (US)) Guillelmo Gomez Ceballos Retuerto (Massachusetts Inst. of Technology (US)) Hannes Sakulin (CERN) James Gordon Branson (Univ. of California San Diego (US)) Mr Jan Veverka (Massachusetts Inst. of Technology (US)) Jean-Marc Olivier Andre (Fermi National Accelerator Lab. (US)) Dr Jeroen Hegeman (CERN) Konstanty Sumorok (Massachusetts Inst. of Technology (US)) Lorenzo Masetti (CERN) Luciano Orsini (CERN) Dr Marc Dobson (CERN) Marco Pieri (Univ. of California San Diego (US)) Olivier Chaze (CERN) Penelope Amelia Roberts (Nottingham Trent University (GB)) Petr Zejdl (CERN) Remi Mommsen (Fermi National Accelerator Lab. (US)) Salvatore Zaza (Sezione di Pisa (IT)) Samim Erhan (Univ. of California Los Angeles (US)) Sergio Cittolin (Univ. of California San Diego (US)) Srecko Morovic (CERN) Tomasz Adrian Bawej (University of Wisconsin (US)) Ulf Behrens (Deutsches Elektronen-Synchrotron (DE)) Vivian O'Dell (Fermi National Accelerator Laboratory (FNAL))

Presentation materials