Transformers excel at symbolic data manipulation, but most of their applications in physics deal with numerical calculations. I present a number of applications of symbolic AI in mathematics, and one in theoretical physics: learning scattering amplitudes.
Informed by the many fields in which machine learning (ML) has made impacts, the coming years promise to see exciting improvements in the discovery and measurement power of LHC experiments. But stepping back from the many exploratory studies ongoing, there are already dozens of concrete and rigorous public LHC results leveraging advanced ML. This review will examine common themes of those...
High-precision simulations based on first principles are a cornerstone of LHC physics research. In view of the HL-LHC era, there is an ever-increasing demand for both accuracy and speed in simulations. In this talk, I will first explain the basic principles of LHC event generation and highlight current methodologies and their bottlenecks. Afterwards, I will delve into the MadNIS journey and...
How can one fully harness the power of physics encoded in relativistic $N$-body phase space? Topologically, phase space is isomorphic to the product space of a simplex and a hypersphere and can be equipped with explicit coordinates and a Riemannian metric. This natural structure that scaffolds the space on which all collider physics events live opens up new directions for machine learning...
Quantum Generative Models are emerging as a promising tool for modelling complex physical phenomena. In this work, we explore the application of Quantum Boltzmann Machines and Quantum Generative Adversarial Networks to the intricate task of jet substructure modelling in high-energy physics. Specifically, we use these quantum frameworks to model the kinematics and corrections of the leading...
Recent innovations from machine learning allow for data unfolding, without binning and including correlations across many dimensions. We describe a set of known, upgraded, and new methods for ML-based unfolding. The performance of these approaches is evaluated on two benchmark datasets. We find that all techniques are capable of accurately reproducing the particle-level spectra across complex...
The Fair Universe project is organising the HiggsML Uncertainty Challenge, which has been running from Sep 2024 to 14th March 2025. It is a NeurIPS 2024 competition.
This HEP and Machine Learning competition is the first to strongly emphasise uncertainties: mastering uncertainties in the input training dataset and outputting credible confidence intervals.
The context is the measurement...
The simplification and reorganization of complex expressions lies at the core of scientific progress, particularly in theoretical high-energy physics. This work explores the application of machine learning to a particular facet of this challenge: the task of simplifying scattering amplitudes expressed in terms of spinor-helicity variables. We demonstrate that an encoder-decoder transformer...
The precise measurement of kinematic features of jets is key to the physics program of the LHC. The determination of the energy and mass of jets containing bottom quarks 𝑏-jets is particularly difficult given their distinct radiation patterns and production of undetectable neutrinos via leptonic heavy flavor decays. This talk will describe a novel calibration technique for the b-jet kinematics...
The High Luminosity upgrade to the LHC will deliver an unprecedented luminosity to the ATLAS experiment. Ahead of this increase in data the ATLAS trigger and data acquisition system will undergo a comprehensive upgrade. The key function of the trigger system is to maintain a high signal efficiency together with a high background rejection whilst adhering to the throughput constraints of the...
The ATLAS experiment reconstructs electrons and photons from clusters of energy deposits in the electromagnetic calorimeter. The reconstructed electron and photon energy must be corrected from the measured energy deposits in the clusters to account for energy loss in passive material upstream of the calorimeter, in the passive material in the calorimeter, out of cluster energies and leakage in...
This talk presents a synergy between quark/gluon jet tagging on LHC data, and charged hadron time-of-flight (TOF) regression on ILC data, in the form of one problem-solving mechanism that can address both tasks. They both involve processing data represented as unordered point clouds of varying sequence lengths, optimally handled using permutation-invariant architectures.
A...
Differentiable programming opens exciting new avenues in particle physics, also affecting future event generators. These new techniques boost the performance of current and planned MadGraph implementations. Combining phase-space mappings with a set of very small learnable flow elements, MadNIS-Lite, can improve the sampling efficiency while being physically interpretable. This defines a third...
Deep learning can give a significant impact on physics performance of electron-positron Higgs factories such as ILC and FCCee. We are working on two topics on event reconstruction to apply deep learning; one is jet flavor tagging. We apply particle transformer to ILD full simulation to obtain jet flavor, including strange tagging. The other one is particle flow, which clusters calorimeter hits...
We attempt to extend the typical stratification of parameter space used during Monte Carlo simulations by considering regions of arbitrary shape. Such regions are defined by directly using their importance for the simulation, for example, a likelihood or scattering amplitude. In particular, we consider the possibility that the parameter space may be high dimensional and the simulation costly...
Identifying the origin of high-energy hadronic jets (`jet tagging') has been a critical benchmark problem for machine learning in particle physics. Jets are ubiquitous at colliders and are complex objects that serve as prototypical examples of collections of particles to be categorized. Over the last decade, machine learning-based classifiers have replaced classical observables as the state...
Background estimation is already a bottleneck in several analyses at LHCb, and with the upcoming larger datasets, the demand for efficient background simulation will continue to grow. While there are existing tools that can provide quick, rough estimates of background reconstructed distributions (e.g. RapidSim), these cannot account for the effects of common selection criteria. The tool...
Attention-based transformer models have become increasingly prevalent in collider analysis, offering enhanced performance for tasks such as jet tagging. However, they are computationally intensive
and require substantial data for training. In this paper, we introduce a new jet classification network
using an MLP mixer, where two subsequent MLP operations serve to transform particle and...
In many real-world scenarios, data is hybrid — i.e. described by both continuous and discrete features. At high-energy accelerators like the LHC, jet constituents exhibit discrete properties such as electric charge or particle-id. In this talk, we introduce a novel generative model for discrete features based on continuous-time Markov jump processes. By combining our approach with well-known...
This study introduces an approach to learning augmentation-independent jet representations using a Jet-based Joint Embedding Predictive Architecture (J-JEPA). This approach aims to predict various physical targets from an informative context, using target positions as joint information. We study several methods for defining the targets and context, including grouping subjets within a jet, and...
AI generative models, such as generative adversarial networks (GANs), have been widely used and studied as efficient alternatives to traditional scientific simulations like Geant4. Diffusion models, which have demonstrated great capability in generating high-quality text-to-image translations in industry, have yet to be applied in the high-energy heavy-ion physics.
In this talk, we present...
Extracting scientific understanding from particle-physics experiments requires solving diverse learning problems with high precision and good data efficiency. We present the Lorentz Geometric Algebra Transformer (L-GATr), a new multi-purpose architecture for high-energy physics. L-GATr represents high-energy data in a geometric algebra over four-dimensional space-time and is equivariant under...
Efficient jet flavour-tagging is crucial for event reconstruction and particle analyses in high energy physics (HEP). Graph Neural Networks (GNNs) excel in capturing complex relationships within graph-structured data, and we aim to enhance the classification of b-jets using this method of deep learning. Presented in this work is the first application of a novel GNN b-jet tagger using the LHCb...
Monte Carlo (MC) simulations are crucial for collider experiments, enabling the comparison of experimental data with theoretical predictions. However, these simulations are computationally demanding, and future developments, like increased event rates, are expected to surpass available computational resources. Generative modeling can substantially cut computing costs by augmenting MC...
One potential roadblock towards the HL-LHC experiment, scheduled to begin in 2029, is the computational demand of traditional collision simulations. Projections suggest current methods will require millions of CPU-years annually, far exceeding existing computational capabilities. Replacing the event showers module in calorimeters with quantum-assisted deep learning surrogates can help bridge...
The steady progress in machine learning leads to substantial performance improvements in various areas of high-energy physics, especially for object identification. Jet flavor identification (tagging) is a prominent benchmark that profits from elaborate architectures, leveraging information from low-level input variables and their correlations. Throughout the data-taking eras of the Large...
Precise tau identification is a crucial component for many studies targeting the Standard Model or searches for New Physics within the CMS physics program. The Deep Tau v2.5 algorithm is a convolutional neural network algorithm: an improved version of its predecessor, Deep Tau v2.1, deployed for the LHC Run 3. This updated version integrates several enhancements to improve classification...
As data sets grow in size and complexity, simulated data play an increasingly important role in analysis. In many fields, two or more distinct simulation software applications are developed that trade off with each other in terms of accuracy and speed. The quality of insights extracted from the data stand to increase if the accuracy of faster, more economical simulation could be improved to...
The phenomena of Jet Quenching, a key signature of the Quark-Gluon Plasma (QGP) formed in Heavy-Ion (HI) collisions, provides a window of insight into the properties of this primordial liquid. In this study, we rigorously evaluate the discriminating power of Energy Flow Networks (EFNs), enhanced with substructure observables, in distinguishing between jets stemming from proton-proton (pp) and...
The CMS Fast Simulation chain (FastSim) is roughly 10 times faster than the application based on the GEANT4 detector simulation and full reconstruction referred to as FullSim. This advantage however comes at the price of decreased accuracy in some of the final analysis observables. A machine learning-based technique to refine those observables has been developed and its status is presented...
Fast event and detector simulation in high-energy physics using generative models provides a viable solution for generating sufficient statistics within a constrained computational budget, particularly in preparation for the High Luminosity LHC. However, many of these applications suffer from a quality/speed tradeoff. Diffusion models offer some of the best sampling quality but slow generation...
We improve upon the existing literature on pileup mitigation techniques studied at Large Hadron Collider (LHC) experiments for disentangling proton-proton collisions. Pileup presents a salient problem that, if not checked, hinders the search for new physics and Standard Model precision measurements such as jet energy, jet substructure, missing momentum, and lepton isolation. The primary...
Simulating particle physics data is an essential yet computationally intensive process in analyzing data from the LHC. Traditional fast simulation techniques often use a surrogate calorimeter model followed by a reconstruction algorithm to produce reconstructed objects. In this work, we introduce Particle-flow Neural Assisted Simulations (Parnassus), a deep learning-based method for generating...
Supervised deep learning methods have found great success in the field of high energy physics (HEP) and the trend within the field is to move away from high level reconstructed variables to low level detector features. However, supervised methods require labelled data, which is typically provided by a simulator. The simulations of HEP datasets become harder to validate and calibrate as we...
We propose a novel framework to obtain asymptotic frequentist uncertainties on machine learned classifier outputs by using model ensembles. With the well-known likelihood trick, this framework can then be applied to the task of density ratio estimation to obtain statistically rigorous frequentist uncertainties on estimated likelihood ratios. As a toy example, we demonstrate that the framework...
Observables sensitive to top quark polarization are important for characterizing and discovering new physics. The most powerful spin analyzer in the top decay is the down-type fermion from the W, which in the case of leptonic decay allows for very clean measurements. However, in many applications, it is useful to measure the polarization of hadronically decaying top quarks via an optimal...
Energy correlators have recently shown potential to improve the precision on the top mass precision measurement. However, existing measurement strategies still only use part of the information in the EEEC distribution and rely on arbitrary shape choices. In this talk, we explore the ability of Machine Learning to effectively optimize shape choice and reduce error on the top mass. Specifically,...
Analysis of collision data often involves training deep learning classifiers on very specific tasks and in regions of phase-space where the training datasets have limited statistics. Models pre-trained on a larger, more generic, sample may already have a useful representation of collider data which can be leveraged by many independent downstream analysis tasks. We introduce a class of...
ATLAS explores modern neural networks for a multi-dimensional calibration of its calorimeter signal defined by clusters of topologically connected cells (topo-clusters). The Bayesian neural network (BNN) approach yields a continuous and smooth calibration function, including uncertainties on the calibrated energy per topo-cluster. In this talk the performance of this BNN-derived calibration is...
Machine learning is becoming increasingly popular in the context of particle physics. Supervised learning, which uses labeled Monte Carlo simulations, remains one of the most widely used methods for discriminating signals beyond the Standard Model. However, this paper suggests that supervised models may depend excessively on artifacts and approximations from Monte Carlo simulations,...
Evidential Deep Learning (EDL) is an uncertainty-aware deep learning approach designed to provide confidence (or epistemic uncertainty) about test data. It treats learning as an evidence acquisition process where more evidence is interpreted as increased predictive confidence. This talk will provide a brief overview of EDL for uncertainty quantification (UQ) and its application to jet tagging...
Large-scale point cloud and long-sequence processing are crucial for high energy physics applications such as pileup mitigation and track reconstruction. The HL-LHC presents inevitable challenges to machine learning models, requiring both high stability and low computational complexity. Previous studies have primarily focused on graph-based approaches which are generally effective but often...
Galactic dynamics studies often face the challenge of incomplete kinematic information in stellar catalogs.
This incompleteness poses a significant challenge to a complete and model-independent measurement of local galactic dark matter densities using stellar dynamics.
This talk presents two innovative approaches that fuse physics principles with machine learning techniques, specifically...
The next decade will see an order of magnitude increase in data collected by high-energy physics experiments,
driven by the High-Luminosity LHC (HL-LHC). The reconstruction of charged particle trajectories (tracks) has
always been a critical part of offline data processing pipelines. The complexity of HL-LHC data will however
increasingly mandate track finding in all stages of an...
Reconstructing particle tracks from detector hits is computationally intensive due to the large combinatorics involved. Recent work has shown that ML techniques can enhance conventional tracking methods, but complex models are often difficult to implement on heterogeneous trigger systems, such as FPGAs. While deploying neural networks on FPGAs is possible, resource limitations pose challenges....
The dynamics of stars in our galaxy encode crucial information about the Milky Way's dark matter halo. However, extinction from foreground dust can bias studies of stellar populations. By solving the equilibrium collisionless Boltzmann equation with novel machine learning techniques, we estimate the unbiased 6-dimensional phase space density of an equilibrated stellar population and the...
Accurately reconstructing particles from detector data is a critical challenge in experimental particle physics, where the spatial resolution of calorimeters plays a key role. This study explores the integration of super-resolution techniques into the Large Hadron Collider (LHC)-like reconstruction pipeline to enhance the granularity of calorimeter data. By applying super-resolution, we...
We introduce SkyCURTAINs, an adaptation of the CURTAINs method—a weakly supervised technique originally developed for anomaly detection in high-energy physics data—applied to data from the second Gaia Data Release (GDR2). SkyCURTAINs is employed to search for stellar streams, which appear as line-like overdensities against the background of the Milky Way. To validate the feasibility of this...
Recreating realistic parton-level event configurations from jets is a crucial task for various physics analyses. However, hadronization processes cannot be computed using perturbative QCD. Therefore, it has been traditionally intractable to reconstruct parton-level events after hadronization.
We present a generative machine learning approach for reconstructing jet showers at the parton...
The major goal of Imaging Atmospheric Cherenkov Telescopes (IACTs) is the investigation of gamma-ray sources through the detection of their induced air showers. For every detected gamma ray, there are up to 10000 cosmic ray protons present forming the background, which also needs to be studied. For a detailed understanding of the instrument for deriving its response to both gamma rays and...
The Large Hadron Collider (LHC) at CERN pushes the boundaries of particle physics, generating data at unprecedented rates and requiring advanced computational techniques to process information in real time. While experimental environments between LHC experiments can differ, common challenges can be identified in the area of real-time reconstruction including the use of specialized trigger...
A calibration of the ATLAS flavor-tagging algorithms using a new calibration procedure based on optimal transportation maps is presented. Simultaneous, continuous corrections to the $b$-, $c$-, and light flavor classification probabilities from jet tagging algorithms in simulation are derived for $b$-jets using $t\bar t \to b \bar b e \mu \nu \nu$ events. After application of the derived...
I report the final results of the Fast Calorimeter Challenge 2022: 23 collaborations submitted 59 samples across all 4 datasets. I will show how these rank regarding various metrics judging shower quality, generation time, and other properties. From these results, I present the current, state-of-the-art, Pareto fronts for using deep generative models on high-dimensional datasets in high-energy...
The development of analysis methods that can distinguish potential beyond the Standard Model phenomena in a model-agnostic way can significantly enhance the discovery reach in collider experiments. However, the typical machine learning (ML) algorithms employed for this task require fixed length and ordered inputs that break the natural permutation invariance in collider events. To address this...
Generative models are on a fast track to becoming a mainstay in particle physics simulation chains, seeing active work towards adoption by nearly every large experiment and collaboration. However, the question of estimating the uncertainties and statistical expressiveness of samples produced by generative ML models is still far from settled.
Recently, combinations of generative and...
There has been significant work recently in developing machine learning (ML) models in high energy physics (HEP) for tasks such as classification, simulation, and anomaly detection. Often these models are adapted from those designed for datasets in computer vision or natural language processing, which lack inductive biases suited to HEP data, such as equivariance to its inherent symmetries....
Recently, Kolmogorov-Arnold Networks (KANs) have been proposed as an alternative to multilayer perceptrons, suggesting advantages in performance and interpretability. In this talk, we present the first application of KANs in high-energy physics, focusing on a typical binary classification task involving high-level features.
We study KANs with different depths and widths and include a...
In the realm of high-energy physics, the use of graph network-based implementations offers the advantage of handling input datasets more closely aligned with their collection process in collider experiments. GNN-based approaches address the graph anomaly detection problem by utilizing information about graph features and structures to effectively learn to score anomalies. We represent a single...
We are presenting the first calibration of the jet pT regression (CMS-DP-2024-064), achieving an expected improvement in jet resolution of up to 17%, and the latest performance results for flavor identification and jet energy resolution estimation using ParticleNet. The pT regression, which focuses on correcting the reconstructed jet pT to the truth-level jet pT, is divided into two...
We introduce a model-agnostic search for new physics in the dijet final state. Other than the requirement of a narrow dijet resonance with a mass in the range of 1.8-6 TeV, minimal additional assumptions are placed on the signal hypothesis. Search regions are obtained by utilizing multivariate machine learning methods to select jets with anomalous substructure. A collection of complementary...
A key step in any resonant anomaly detection search is accurate estimation of the background distribution in each signal region. Data-driven methods like CATHODE accomplish this by training separate density estimators on the complement of each signal region, and interpolating them into their corresponding signal regions. Having to re-train the density estimator on essentially the entire...
We introduce TRANSIT, a conditional adversarial network for continuous interpolation of data. It is designed to construct a background data template for semi-supervised searches for new physics processes at the LHC, by smoothly transforming sideband events to match signal region mass distributions.
We demonstrate the performance of TRANSIT using the LHC Olympics R&D dataset. The method...
To compare collider experiments, measured data must be corrected for detector distortions through a process known as unfolding. As measurements become more sophisticated, the need for higher-dimensional unfolding increases, but traditional techniques have limitations. To address this, machine learning-based unfolding methods were recently introduced. In this work, we introduce OmniFoldHI, an...
Machine learning-based unfolding has started to establish itself as the go-to approach for precise, high-dimensional unfolding tasks. The current state-of-the-art unfolding methods can be divided into reweighting-based and generation-based methods. The latter of the two is comprised of conditional generative models, which generate new truth-level events from random noise conditioned on...
Measurements of jet substructure are key to probing the energy frontier at colliders, and many of them use track-based observables which take advantage of the angular precision of tracking detectors. Theoretical calculations of track-based observables require “track functions”, which characterize the transverse momentum fraction $r_q$ carried by charged hadrons from a fragmenting quark or...
The measurements performed by particle physics experiments must account for the imperfect response of the detectors used to observe the interactions. One approach, unfolding, statistically adjusts the experimental data for detector effects. Recently, generative machine learning models have shown promise for performing unbinned unfolding in a high number of dimensions. However, all current...
Simulating showers of particles in highly-granular detectors is a key frontier in the application of machine learning to particle physics. Achieving high accuracy and speed with generative machine learning models can enable them to augment traditional simulations and alleviate a major computing constraint.
Recent developments have shown how diffusion based generative shower simulation...
Many physics analyses at the LHC rely on algorithms to remove detector effect, commonly known as unfolding. Whereas classical methods only work with binned, one-dimensional data, Machine Learning promises to overcome both problems. Using a generative unfolding pipeline, we show how it can be build into an existing LHC analysis, designed to measure the top mass. We discuss the model-dependence...
We present an application of Simulation-Based Inference (SBI) in collider physics, aiming to constrain anomalous interactions beyond the Standard Model (SM). This is achieved by leveraging Neural Networks to learn otherwise intractable likelihood ratios. We explore methods to incorporate the underlying physics structure into the likelihood estimation process. Specifically, we compare two...
We propose a new approach to learning powerful jet representations directly from unlabelled data. The method employs a Particle Transformer to predict masked particle representations in a latent space, overcoming the need for discrete tokenization and enabling it to extend to arbitrary input features beyond the Lorentz four-vectors. We demonstrate the effectiveness and flexibility of this...
Machine learning has become an essential tool in jet physics. Due to their complex, high-dimensional nature, jets can be explored holistically by neural networks in ways that are not possible manually. However, innovations in all areas of jet physics are proceeding in parallel. We show that large machine learning models trained for a jet classification task can improve the accuracy, precision,...
Correcting for detector effects in experimental data, particularly through unfolding, is critical for enabling precision measurements in high-energy physics. However, traditional unfolding methods face challenges in scalability, flexibility, and dependence on simulations. We introduce a novel approach to multidimensional particle-wise unfolding using conditional Denoising Diffusion...
OmniJet-alpha is the first cross-task foundation model for particle physics, demonstrating transfer learning between an unsupervised problem (jet generation) and a classic supervised task (jet tagging). While OmniJet-alpha is still at a prototype stage, the successful development of foundation models for physics data would represent a major breakthrough, as they have the potential to enhance...
Neural Simulation-Based Inference (NSBI) is a powerful class of machine learning (ML)-based methods for statistical inference that naturally handle high dimensional parameter estimation without the need to bin data into low-dimensional summary histograms. Such methods are promising for a range of measurements at the Large Hadron Collider, where no single observable may be optimal to scan over...
This study proposes a new method for training foundation models designed explicitly for jet-related tasks. Like those seen in large language models, a foundation model is a pre-trained model that can be fine-tuned for various applications and is not limited to a specific task. Previous approaches often involve randomly masking inputs, such as tracks within a jet, and then predicting the masked...
Determining the form of the Higgs potential is one of the most exciting challenges of modern particle physics. Higgs pair production directly probes the Higgs self-coupling and should be observed in the near future at the High-Luminosity LHC. We explore how to improve the sensitivity to physics beyond the Standard Model through per-event kinematics for di-Higgs events. In particular, we employ...
This study introduces an innovative approach to analyzing unlabeled data in high-energy physics (HEP) through the application of self-supervised learning (SSL).
Faced with the increasing computational cost of producing high-quality labeled simulation samples at the CERN LHC, we propose leveraging large volumes of unlabeled data to overcome the limitations of supervised learning methods, which...
The high-luminosity era of the LHC will pose unprecedented challenges to the detectors. To meet these challenges, the CMS detector will undergo several upgrades, including the replacement the current endcap calorimeters with a novel High-Granularity Calorimeter (HGCAL). To make optimal use of this innovative detector, novel algorithms have to be invented. A dedicated reconstruction framework,...
Weakly supervised anomaly detection has been shown to have great potential for improving traditional resonance searches. We demonstrate that weak supervision offers a unique opportunity to turn a resonance search into a simple cut-and-count experiment, where the potential problem of background sculpting in a traditional bump hunt is absent. Moreover, the cut-and-count setting allows working...
In this talk I will give a biased review on the work at the intersection of machine learning and theoretical physics. This includes how we can use transformers to obtain symbolic expressions without having information about the target expression. In turn, I present a benchmark human physicists have failed in solving, namely that of compact Calabi-Yau metrics and give a short status report on...
