Speaker
Description
Unsupervised anomaly detection has become a pivotal technique for model-independent searches for new physics at the LHC. In high-energy physics (HEP), anomaly detection is employed to identify rare, outlier events in collision data that deviate significantly from expected distributions. A promising approach is the application of generative machine learning models, which can efficiently detect such deviations without requiring labeled data.
In this study, we develop a Transformer-based reconstruction model, trained exclusively on Standard Model (SM) background data, to identify events that exhibit significant deviations. The method is applied to ATLAS Open Data from Run 2 (2015–2016), focusing on the identification of rare and potential Beyond the Standard Model (BSM) processes. Our architecture utilizes a modified Transformer, optimized to handle high-dimensional tabular input, comprising low-level physics observables, such as jet kinematics, lepton and photon energy, MET, electromagnetic and hadronic calorimeter energy deposits, as well as event topology variables.
The Transformer model is trained to learn the inherent patterns in SM background data, effectively modeling the normal event distributions. We use a Tab-Transformer with weighted loss, which captures the intricate relationships within the background data. When the trained model is tested on rare and BSM Monte Carlo (MC) samples (e.g., SUSY, Exotic), it exhibits excellent reconstruction performance for background events while generating large reconstruction losses for anomalous events. This ability to identify outliers is crucial for anomaly detection in HEP.
Compared to conventional Variational Autoencoders (VAEs), our Transformer-based architecture demonstrates superior background modeling, with enhanced sensitivity to anomalies. The method operates directly on low-level physics observables, making it highly interpretable and scalable. Additionally, it allows for searches in pre-selection regions without introducing biases from selection cuts, offering a more flexible approach to identifying new physics. We are also planning to extend the analysis using more detector-level observables to further improve the sensitivity and scalability of the method.
References
https://doi.org/10.1140/epjc/s10052-024-12527-w
Significance
This is a significant methodological advance on model-independent new physics searches in that it demonstrates that transformer-based architectures can outperform conventional approaches at detecting subtle deviations in high-dimensional low-level HEP data. This opens a novel path for unsupervised anomaly detection at the LHC, enabling early-stage discovery potential in regions typically inaccessible to traditional analysis. By operating on detector-level observables directly and achieving more generalization to unknown signals, our study helps develop interpretable, scalable AI tools that enhance the sensitivity and adaptability of upcoming collider studies. Furthermore, the full code and documentation will be made publicly available on GitHub, supporting transparency, collaboration, and wider use of AI-based methods in HEP. This enables optimized access to and utilization of Open Data, further entrenching the role of reproducible machine learning pipelines in upcoming experimental research.
