Speaker
Description
Data Quality Monitoring (DQM) is a very significant component of all high-
energy physics (HEP) experiments. Data recorded by Data Acquisition (DAQ) sensors and devices are sampled to perform live monitoring of the status of each detector during data collection. This gives to the system and scientists the ability to identify problems with extremely low latency, minimizing the amount of data that would otherwise be unsuitable for physical analysis. In the offline DQM environment, the system is used to review the results of the final reconstruction of data on a continuous baseline, serving as the certified database used in all scientific analysis. DQM performs a large set of operations on the data, such as Fast Fourier Transform (FFT), Clustering algorithms, Region of Interest (RoI) or Classification problems, for instance. All those operations suppose an intensive processing workflow with massive volumes of data to be performed on High Performance Computing (HPC) platforms.
This workflow involves the use of HPC resources and time, both depending on
the number of events of the experiment, the complexity of the tasks or the data quality levels, among others. The use of Machine Learning (ML) techniques in the DQM stage can significantly improve the general performance in the execution of these workflows, achieving a better performance in the management and distribution of the processes that run on HPC.
In this context,the main objective of our work is the application of ML algorithms in order to improve the use of HPC resources and increase the overall quality levels of the data processed in DQM. As an effective solution a Multi-Objective Evolutionary Algorithm (MOEA) has been proposed to accomplish this optimization, considering objectives, resources and constraints. MOEA has been used due
to its good balance between computational cost and quality solutions reached.
Finally a tool for decision making has been deployed allowing the scientists to
decide more efficiently on DQM process.
