Indico has been upgraded to version 3.1. Details in the SSB
Nov 4 – 8, 2019
Adelaide Convention Centre
Australia/Adelaide timezone

Big Data solutions for the online processing of trigger-less detectors data

Nov 5, 2019, 5:00 PM
Riverbank R4 (Adelaide Convention Centre)

Riverbank R4

Adelaide Convention Centre

Oral Track X – Crossover sessions from online, offline and exascale Track X – Crossover sessions


Marco Zanetti (Universita e INFN, Padova (IT))


The need for an unbiased analysis of large complex datasets, especially those collected by the LHC experiments, is pushing for data acquisition systems where predefined online trigger selections are limited if not suppressed at all. Not just this poses tremendous challenges for the hardware components, but also calls for new strategies for the online software infrastructures. Open source Big-Data tools could certainly offer valuable solutions for the latter.
In view of the high luminosity upgrade of the LHC, we developed a prototype online processing scheme for the CMS muon detectors (Drift Tubes Chambers, DT), which streams signals from the front-end electronics at the same rate as the LHC clock (40 MHz) and serves them through Apache Kafka to a remote Apache Spark cluster where offline quality reconstruction algorithms are run. Extensive tests have been carried out demonstrating the scalability of the system, in particular the throughput expected from the DT chambers at HL-LHC can be sustained by a computing cluster of comparable size as the current prototype. This setup has been exploited successfully for beam tests and will be deployed for parasitic operations in CMS during the next LHC Run.

Consider for promotion No

Primary authors

Marco Zanetti (Universita e INFN, Padova (IT)) Jacopo Pazzini (Universita e INFN, Padova (IT)) Matteo Migliorini (Universita e INFN, Padova (IT))

Presentation materials