Bites of FM4S: [1] Physics-inspired representations

Wednesday 20 Nov 2024, 14:00 → 17:40 Europe/Zurich

Following up on the Foundation Models for Science mini-workshop, the CMS ML innovation team is excited to announce Bites of Foundation Models for Science, deep diving into specific themes in this area.

 The first of these bites will be held remotely on Nov. 20th 2024 (14:00-18:00 CERN time) with the theme of learning representations inspired by and aimed for physics applications. We aim to connect researchers from CMS, ATLAS, and beyond HEP working on building powerful, robust representations for science.

 We invite you to share your work! Abstract submission for the first workshop is open until November 13th. 

 We plan to follow-up with the next themes of discussion every 1 - 2 months. Please register below to receive updates on the future events. Abstract submission for these is open as well. Feel free to submit an abstract for a theme that better fits your work, we will keep it in mind for future events!

 Don’t hesitate to contact us at cms-conveners-ml-innovation@cern.ch for more details or questions!

 

Organizing committee:

• Javier Duarte (UCSD)
• Gaia Grosso (IAIFI / MIT)
• Raghav Kansal (Caltech / FNAL)
• Pietro Vischia (Universidad de Oviedo / ICTEA)

14:00 → 14:10 Introduction to Bites of FM4S

Speakers: Gaia Grosso (IAIFI, MIT), Raghav Kansal (California Institute of Technology)

14:10 → 15:00 Boosting the LHC resonance search program with Sophon

We introduce a novel experimental methodology, Signature-Oriented Pre-training for Heavy-resonance ObservatioN (Sophon), designed to enhance the LHC resonance search program in the Lorentz-boosted regime. Sophon leverages the principles of “large models for large-scale classification”, employing the advanced deep learning algorithm to train a classifier across an extensive (o(100)) set of boosted final states provided by the newly developed JetClass-II dataset. We show that the resulting model (the Sophon model) is capable of learning intricate jet signatures, achieving two key objectives: (1) optimal constructions of various jet tagging discriminates and (2) high-performance transfer learning capabilities across new tasks. These capabilities ensure Sophon pushes widespread model-specific searches to their sensitivity frontier and also significantly improves model-agnostic approaches, thereby accelerating LHC resonance searches in a broad sense.

Speaker: Congqiao Li (Peking University (CN))

24.11.20_FM4S_Sophon.pdf

GMT20241120-130927_Recording_1792x1120.mp4

15:00 → 15:20 Anomaly preserving neural embeddings for New Physics discovery at the LHC

Discovering the occurrence of unexpected physical processes in collider data could unveil new fundamental laws governing our Universe. However, the extreme size, rate and complexity of the datasets generated at the Large Hadron Collider (LHC) pose unique challenges to detect them. Typically, this is addressed by transforming high-dimensional, low-level detector data into physically meaningful summary statistics, like particle invariant masses. In this work we address the problem of data reduction for anomaly detection by learning powerful new lower dimensional representations of the data via neural embeddings that preserve the anomalous features.
We consider synthetic LHC data originally represented by the kinematic variables of 19 physics objects produced in a collision event, and we study different MLP- and Transformer-based neural embeddings trained according to supervised or self-supervised contrastive learning methods. The learnt embeddings are used as input representation to signal-agnostic statistical detection tools, showing increased detection performances compared to the use of the original representation.

Speaker: Kyle Sidney Metzger (ETH Zurich (CH))

video1783558732.mp4

Workshop slides.pdf

15:20 → 15:40 Rapid Parameter Estimation for Kilonovae Using Likelihood-Free Inference

Rapid parameter estimation is critical when dealing with short lived signals such as kilonovae. We present a parameter estimation algorithm that combines likelihood-free inference with a pre-trained embedding network, optimized to efficiently process kilonova light curves. Our method is capable of retrieving two intrinsic parameters of the kilonova light curves with a comparable accuracy and precision to nested sampling methods while taking significantly less computational time. Our inference uniquely utilizes a pre-trained embedding network that marginalizes the time of arrival and the luminosity distance of the signal, allowing inference of signals at distances up to 200 Mpc. We find that including a pre-trained embedding outperforms the use of likelihood-free inference alone, reducing training time, model size, and offering the capability to marginalize over certain nuisance parameters. This framework has been integrated into the publicly available Nuclear Multi-Messenger Astronomy codebase so users can deploy the model for their inference purposes. Our algorithm is broadly applicable to parameterized or simulated light curves of other transient objects, and can be adapted for quick sky localization.

Speaker: Malina Desai

FM4S Bites Presentation.pdf

video2783558732.mp4

15:40 → 15:50 break

15:50 → 16:40 You can observe a lot by just watching (new AI strategies for the LHC and beyond)

With large amounts of data, a Higgs boson discovery, and world-leading constraints on an enormous amount of parameters and interactions, the Large Hadron Collider has been a phenomenal tool. We show new results built on contrastive learning and semi-supervised learning strategies where, through physics-motivated choices, we teach an AI to visualize many processes simultaneously, allowing it to solve a variety of downstream tasks in one algorithm. The implications are far-reaching. We discuss how this style of learning broadly allows us to simplify problems, and we point to new directions in representation learning that aim to further unite physics with ML.

Speaker: Philip Coleman Harris (Massachusetts Inst. of Technology (US))

PCH_RepresentationL_19_11.pdf

video3783558732.mp4

16:40 → 17:00 MACK: Mismodeling Addressed with Contrastive Knowledge

The use of machine learning methods in high energy physics typically relies on large volumes of precise simulation for training. As machine learning models become more complex they can become increasingly sensitive to differences between this simulation and the real data collected by experiments. We present a generic methodology based on contrastive learning which is able to greatly mitigate this negative effect and generate expressive representations that are insensitive to simulation specifics. Crucially, the method does not require prior knowledge of the specifics of the mismodeling. While we demonstrate the efficacy of this technique using the task of jet-tagging at the Large Hadron Collider, it is applicable to a wide array of different tasks both in and out of the field of high energy physics.

Speaker: Dylan Sheldon Rankin (University of Pennsylvania (US))

fm4s_dsr_mack_20nov24.pdf

video4783558732.mp4

17:00 → 17:20 Product Manifold Machine Learning for Physics

Particle jets are collimated flows of partons which evolve into tree-like structures through stochastic parton showering and hadronization. The hierarchical nature of particle jets aligns naturally with hyperbolic space, a non-Euclidean geometry that captures hierarchy intrinsically. To leverage the benefits of non-Euclidean geometries, we develop jet analysis in product manifold (PM) spaces, Cartesian products of constant curvature Riemannian manifolds. We consider particle representations as configurable parameters and compare the performance of PM multilayer perceptron models across several possible representations. We find product manifold representations perform equal or better in particle jet classification than fully Euclidean models of the same latent dimension and the same approximate number of parameters. These findings reinforce the view of optimizing geometric representations as a key parameter in maximizing both performance and efficiency.

Speaker: Nathaniel Sherlock Woodward (Massachusetts Inst. of Technology (US))

MiniFoundation_Talk_PMML_Nate_W_11_20_24.pdf

MiniFoundation_Talk_PMML_Nate_W_11_20_24.pptx

video5783558732.mp4