Speaker
Description
Data quality monitoring (DQM) in high-energy physics (HEP) experiments is essential and widely implemented in most large experiments. It provides important real-time information during the commissioning and production phases that allows the early identification of potential issues and eases their resolution.
Existing and performant solutions for online monitoring exist for large experiments such as CMS [1], ALICE [2] and others. For small setups, proprietary solutions such as LabView provide tools to implement it but fail on scaling to high-throughput experiments.
The present work reports on the new online monitoring capabilities of the software Pyrame [3], an open-source framework designed for HEP applications. Pyrame provides an easy-to-deploy solution for command, control and data-acquisition of particle detectors and related test-benches.
Pyrame’s new online monitoring architecture is based on the distribution of the data treatment operations among any module of the system, with multiple input and output (I/O) streams. A common problem in such systems is that different treatment speeds can lead to uncontrolled data loss. To avoid such situation we implement a mechanism that breaks the real-time constraint, at any treatment level, by buffering data and distributing it at the speed of the consumers (potentially leading to subsampling in the worst case). In addition to the distributed data treatment capabilities, Pyrame includes a performance-oriented module dedicated to real-time data acquisition, capable of handling and storing data at 4Gb/s for further treatment.
This approach allows us to use the same code for online and offline data treatment. In the latter case, subsampling is simply forbidden. Legacy ROOT, R, or Python/Panda offline data treatment codes can be adapted to be fed by a file in offline mode or by one of the available online data sources.
The distribution of the data treatment chain over any module allows us to use a modular design. Data filtering operations can be processed first, providing cut or tagged data with a first level of grouping (block level). The resulting data can then be used in concentrating modules, such as multiplexers (e.g.: from multiple detection layers). Finally, the output can be used in higher level treatment processes like online event-builders/reconstruction modules.
References:
[1] Data acquisition software for the CMS strip tracker, doi:10.1088/1742-6596/119/2/022008
[2] The ALICE Data Quality Monitoring System doi:10.1109/RTC.2009.5322152
[3] Pyrame, a rapid-prototyping framework for online systems doi:10.1088/1742-6596/664/8/082028
| Primary Keyword (Mandatory) | Data processing workflows and frameworks/pipelines |
|---|---|
| Secondary Keyword (Optional) | DAQ |
