19–25 Oct 2024
Europe/Zurich timezone

Development of machine-learning based app for anomaly detection in CMSWEB

22 Oct 2024, 17:09
18m
Large Hall B

Large Hall B

Talk Track 7 - Computing Infrastructure Parallel (Track 7)

Speaker

Muhammad Imran (National Centre for Physics (PK))

Description

The CMS experiment's operational infrastructure hinges significantly on the CMSWEB cluster, which serves as the cornerstone for hosting a multitude of services critical to the data taking and analysis. Operating on Kubernetes ("k8s") technology, this cluster powers over two dozen distinct web services, including but not limited to DBS, DAS, CRAB, WMarchive, and WMCore.

In this talk, we propose and develop an application which is specifically tailored to the task of anomaly detection within this ecosystem of services. The core approach involves harnessing the capabilities of machine/deep learning methods, alongside a comprehensive exploration of various service parameters, to identify irregularities and potential threats effectively. The application is designed with the goal that continually monitors these services for any deviations from their expected behavior. Leveraging diverse machine/deep learning techniques and scrutinizing service-specific parameters, the application will be equipped to discern anomalies and aberrations that might signify security breaches or performance issues. Once an anomaly is detected, the system will not only record this event but will also promptly generate alerts. These alerts will be intelligently routed to the relevant service developers or administrators responsible for maintaining the affected components. This proactive alerting mechanism ensures that any emerging issues are swiftly addressed, minimizing potential disruptions and fortifying the overall reliability of the CMSWEB cluster and its critical services.

Primary author

Mr Nasir Hussain (CERN)

Co-authors

Amna Muzaffar (Quaid-i-Azam University) Muhammad Imran (National Centre for Physics (PK)) Andreas Pfeiffer (CERN) Aroosha Pervaiz (National Centre for Physics (PK)) Valentin Y Kuznetsov (Cornell University (US))

Presentation materials