Login

Foundation Models for Science Mini Workshop

1–2 Oct 2024
Europe/Zurich timezone

Contribution List

14 out of 14 displayed
Export to PDF
  1. 3. Introduction
    Dr Gaia Grosso (IAIFI, MIT), Raghav Kansal (Univ. of California San Diego (US))
    01/10/2024, 14:00
  2. 7. Towards Foundation Models in HEP with Self-Supervised Learning
    Michael Kagan (SLAC National Accelerator Laboratory (US))
    01/10/2024, 14:10

    Can Foundation Models, which rely on massive parameter counts, data sets, and compute, and have proven extremely powerful in computer vision and natural language systems, be built for High Energy Physics? To do so, several challenges must be addressed, including understanding how the training strategies, which are often data-type specific, can be developed for HEP data. In this talk, we...

    Go to contribution page
  3. 14. Automatic Differentiation in RooFit and Community Needs
    Jonas Rembser (CERN), Vassil Vasilev (Princeton University (US)), Vassil Vassilev (CERN)
    01/10/2024, 15:00
  4. 2. Foundation models for HEP
    Philip Coleman Harris (Massachusetts Inst. of Technology (US))
    01/10/2024, 15:45
  5. 6. Invisible Cities: Towards a multi-modal era of fundamental physics research
    Mariel Pettee (Lawrence Berkeley National Lab. (US))
    01/10/2024, 16:30

    To achieve some of the biggest physics discoveries in the last decade -- e.g. finding definitive evidence of the Higgs boson, gravitational waves, and black holes -- physicists had to radically re-imagine the paradigm of working in small teams and instead construct large-scale experimental collaborations of hundreds or even thousands of scientists. The recent success of foundation models in...

    Go to contribution page
  6. 11. Foundation Models for Scientific Discovery in the Lab of the Future
    Patrick Ruch
    01/10/2024, 17:30

    We reflect on the changes imparted by foundation models (FMs) to data-driven exploration and discovery in traditional scientific fields such as chemistry, spanning from hypothesis generation to experimental planning and validation. The emergence of multi-modal FMs also opens up new opportunities for data capture during manual experimentation, for which we present examples and lessons learned....

    Go to contribution page
  7. 4. Introduction to Day 2
    Dr Gaia Grosso (IAIFI, MIT), Raghav Kansal (Univ. of California San Diego (US))
    02/10/2024, 14:00
  8. 9. Re-simulation-based self-supervised learning
    Benedikt Maier (Imperial College (GB))
    02/10/2024, 14:15

    Self-Supervised Learning (SSL) is at the core of training modern large machine learning models, providing a scheme for learning powerful representations that can be used in a variety of downstream tasks. We propose RS3L ("Re-simulation-based self-supervised representation learning"), a novel simulation-based SSL strategy that employs a method of re-simulation to drive data augmentation for...

    Go to contribution page
  9. 8. Finetuning Foundation Models for Joint Analysis Optimization
    Nicole Michelle Hartman (TUM (DE))
    02/10/2024, 15:00

    This talk highlights the significant gains in performance and data efficiency that can be achieved in HEP by moving away from the standard paradigm of separate reconstruction and analysis optimization. We introduce the key idea of fine-tuning a foundation model as a generalization of choosing working points in a physics analysis. The sensitivity gains achievable from end-to-end pipelines are...

    Go to contribution page
  10. 5. OmniJet-α: The first cross-task foundation model for particle physics
    Joschka Birk (Hamburg University (DE))
    02/10/2024, 16:00

    Foundation models are multi-dataset and multi-task machine learning methods that once pre-trained can be fine-tuned for a large variety of downstream applications. The successful development of such general-purpose models for physics data would be a major breakthrough as they could improve the achievable physics performance while at the same time drastically reduce the required amount of...

    Go to contribution page
  11. 10. Foundation Models for Astrophysics and Representation Learning
    Leopoldo Sarra
    02/10/2024, 16:45

    Given the remarkable success of foundation models in language and vision, it is worth exploring whether a similar approach can be applied to scientific domains. These models have the potential to improve computational efficiency, generalize better to low-data regimes, and significantly amortize training costs. However, many questions remain open regarding architectures, data selection,...

    Go to contribution page
  12. 1. OmniLearn: A Method to Simultaneously Facilitate All Jet Physics Tasks
    Vinicius Massami Mikuni (Lawrence Berkeley National Lab. (US))
    02/10/2024, 17:30

    Machine learning has become an essential tool in jet physics. Due to their complex, high-dimensional nature, jets can be explored holistically by neural networks in ways that are not possible manually. However, innovations in all areas of jet physics are proceeding in parallel. We show that specially constructed machine learning models trained for a specific jet classification task can...

    Go to contribution page
  13. 13. Summary and Roundtable
    Dr Gaia Grosso (IAIFI, MIT), Raghav Kansal (Univ. of California San Diego (US))
    02/10/2024, 18:30
  14. 12. Talk