Can Foundation Models, which rely on massive parameter counts, data sets, and compute, and have proven extremely powerful in computer vision and natural language systems, be built for High Energy Physics? To do so, several challenges must be addressed, including understanding how the training strategies, which are often data-type specific, can be developed for HEP data. In this talk, we...
To achieve some of the biggest physics discoveries in the last decade -- e.g. finding definitive evidence of the Higgs boson, gravitational waves, and black holes -- physicists had to radically re-imagine the paradigm of working in small teams and instead construct large-scale experimental collaborations of hundreds or even thousands of scientists. The recent success of foundation models in...
Self-Supervised Learning (SSL) is at the core of training modern large machine learning models, providing a scheme for learning powerful representations that can be used in a variety of downstream tasks. We propose RS3L ("Re-simulation-based self-supervised representation learning"), a novel simulation-based SSL strategy that employs a method of re-simulation to drive data augmentation for...
This talk highlights the significant gains in performance and data efficiency that can be achieved in HEP by moving away from the standard paradigm of separate reconstruction and analysis optimization. We introduce the key idea of fine-tuning a foundation model as a generalization of choosing working points in a physics analysis. The sensitivity gains achievable from end-to-end pipelines are...
Foundation models are multi-dataset and multi-task machine learning methods that once pre-trained can be fine-tuned for a large variety of downstream applications. The successful development of such general-purpose models for physics data would be a major breakthrough as they could improve the achievable physics performance while at the same time drastically reduce the required amount of...
Machine learning has become an essential tool in jet physics. Due to their complex, high-dimensional nature, jets can be explored holistically by neural networks in ways that are not possible manually. However, innovations in all areas of jet physics are proceeding in parallel. We show that specially constructed machine learning models trained for a specific jet classification task can...
