21–25 May 2012
New York City, NY, USA
US/Eastern timezone

Agents and Daemons, automating Data Quality Monitoring operations.

24 May 2012, 13:30
4h 45m
Rosenthal Pavilion (10th floor) (Kimmel Center)

Rosenthal Pavilion (10th floor)

Kimmel Center

Poster Software Engineering, Data Stores and Databases (track 5) Poster Session

Speaker

Luis Ignacio Lopera Gonzalez (Universidad de los Andes (CO))

Description

Since 2009 when the LHC came back to active service, the Data Quality Monitoring (DQM) team was faced with the need to homogenize and automate operations across all the different environments within which DQM is used for data certification. The main goal of automation is to reduce operator intervention at the minimum possible level, especially in the area of DQM files management, where long-term archival presented the greatest challenges. Manually operated procedures cannot cope with the constant increase in luminosity, datasets and time of operation of the CMS detector. Therefore a solid and reliable set of agents has been designed since the beginning to manage all DQM-data related work-flows. This allows to fully exploit all available resources in every condition, maximizing the performance and reducing the latency in making data available for validation and certification. The agents can be easily fine-tuned to adapt to current and future hardware constraints and they proved to be flexible enough to include unforeseen features, like an ad-hoc quota management and a real time sound alarm system.

Primary author

Luis Ignacio Lopera Gonzalez (Universidad de los Andes (CO))

Presentation materials