Preparing distributed computing operations for the HLLHC era with Operational Intelligence

17 May 2021, 16:40
30m
Long talk Distributed Computing, Data Management and Facilities Monday PM plenaries

Speaker

Panos Paparrigopoulos (CERN)

Description

The Operational Intelligence (OpInt) project is a joint effort from
various WLCG communities aimed at increasing the level of automation
in computing operations and reducing human interventions. The currently deployed systems have proven to be mature and capable of meeting the experiments goals, by allowing timely delivery of scientific results. However, a substantial number of interventions from software developers, shifters and operational teams is needed to manage efficiently such heterogeneous infrastructures.
Under the scope of the OpInt project, experts from most of the relevant areas
have gathered to propose and work on “smart” solutions. Machine learning,
data mining, log analysis, and anomaly detection are only some of the tools we
have evaluated for our use cases . Discussions have led to a number of ideas on
how to achieve our goals and the development of solutions has started. In this
contribution, we will report on the development of a suite of OpInt services to
cover various use cases of: workload management, data management, and site
operations.

Primary authors

Panos Paparrigopoulos (CERN) Alessandro Di Girolamo (CERN) Federica Legger (Universita e INFN Torino (IT)) Jaroslava Schovancova (CERN) Thomas Beermann (Bergische Universitaet Wuppertal (DE)) Michael Boehler (Albert Ludwigs Universitaet Freiburg (DE)) Prof. Daniele Bonacorsi (University of Bologna) Luca Clissa (Universita e INFN, Bologna (IT)) Dr Leticia Decker De Sousa (Universita e INFN, Bologna (IT)) Tommaso Diotalevi (Universita e INFN, Bologna (IT)) Luca Giommi (Universita e INFN, Bologna (IT)) Domenico Giordano (CERN) David Hohn (Albert Ludwigs Universitaet Freiburg (DE)) Tomas Javurek (CERN) Stephane Jezequel (LAPP-Annecy CNRS/USMB (FR)) Valentin Y Kuznetsov (Cornell University (US)) Mario Lassnig (CERN) Siarhei Padolski (BNL) Lorenzo Rinaldi (Universita e INFN, Bologna (IT)) Mayank Sharma (University of Texas at Arlington (US)) Simone Rossi Tisbeni (INFN - National Institute for Nuclear Physics) Tuckus Nikodemas (Vilnius University (LT)) Dr Maria Grigorieva (Moscow State University, Moscow, Russia) Ms Micol Olocco (INFN Turin, Italy) Mr Vasilis Mageirakos (CERN, Geneva, Switzerland)

Presentation materials