Speaker
Srecko Morovic
(CERN)
Description
A flexible monitoring system has been designed for the CMS File-based Filter Farm making use of modern data mining and analytics components. All the metadata and monitoring information concerning data flow and execution of the HLT are generated locally in the form of small “documents” using the JSON encoding. These documents are indexed into a hierarchy of elasticsearch (es) clusters along with process and system log information. Elasticsearch is a search server based on Apache Lucene. It provides a distributed, multitenant-capable search and aggregation engine. Since es is schema-free, any new information can be added seamlessly and the unstructured information can be queried in non-predetermined ways. The leaf es clusters consist of the very same nodes that form the Filter Farm thus providing “natural” horizontal scaling. A separate “central" es cluster is used to collect and index aggregated information. The fine-grained information, all the way to individual processes, remains available in the leaf clusters. The central es cluster provides quasi-real-time high-level monitoring information to any kind of client. Historical data can be retrieved to analyse past problems or correlate them with external information. We discuss the design and performance of this system in the context of the CMS DAQ commissioning for LHC Run 2.
Primary authors
Emilio Meschi
(CERN)
Srecko Morovic
(CERN)
Co-authors
Anastasios Andronidis
(Aristotle Univ. of Thessaloniki (GR))
Andre Georg Holzner
(Univ. of California San Diego (US))
Andrea Petrucci
(CERN)
Dr
Attila Racz
(CERN)
Aymeric Arnaud Dupont
(CERN)
Benjamin Stieger
(CERN)
Carlos Nunez Barranco Fernandez
(CERN)
Christian Deldicque
(CERN)
Christoph Paus
(Massachusetts Inst. of Technology (US))
Christoph Schwick
(CERN)
Dominique Gigi
(CERN)
Frank Glege
(CERN)
Frans Meijers
(CERN)
Georgiana Lavinia Darlea
(Massachusetts Inst. of Technology (US))
Guillelmo Gomez Ceballos Retuerto
(Massachusetts Inst. of Technology (US))
Hannes Sakulin
(CERN)
James Gordon Branson
(Univ. of California San Diego (US))
Mr
Jan Veverka
(Massachusetts Inst. of Technology (US))
Jean-Marc Olivier Andre
(Fermi National Accelerator Lab. (US))
Dr
Jeroen Hegeman
(CERN)
Konstanty Sumorok
(Massachusetts Inst. of Technology (US))
Lorenzo Masetti
(CERN)
Luciano Orsini
(CERN)
Dr
Marc Dobson
(CERN)
Marco Pieri
(Univ. of California San Diego (US))
Olivier Chaze
(CERN)
Penelope Amelia Roberts
(Nottingham Trent University (GB))
Petr Zejdl
(CERN)
Remi Mommsen
(Fermi National Accelerator Lab. (US))
Salvatore Zaza
(Sezione di Pisa (IT))
Samim Erhan
(Univ. of California Los Angeles (US))
Sergio Cittolin
(Univ. of California San Diego (US))
Tomasz Adrian Bawej
(University of Wisconsin (US))
Ulf Behrens
(Deutsches Elektronen-Synchrotron (DE))
Vivian O'Dell
(Fermi National Accelerator Laboratory (FNAL))
