11–15 Mar 2024
Charles B. Wang Center, Stony Brook University
US/Eastern timezone

Finetuning Foundation Models for Joint Analysis Optimization

12 Mar 2024, 12:10
20m
Lecture Hall 2 ( Charles B. Wang Center, Stony Brook University )

Lecture Hall 2

Charles B. Wang Center, Stony Brook University

100 Circle Rd, Stony Brook, NY 11794
Oral Track 2: Data Analysis - Algorithms and Tools Track 2: Data Analysis - Algorithms and Tools

Speaker

Matthias Vigl (Technische Universitat Munchen (DE))

Description

In this work we demonstrate that significant gains in performance and data efficiency can be achieved moving beyond the standard paradigm of sequential optimization in High Energy Physics (HEP). We conceptually connect HEP reconstruction and analysis to modern machine learning workflows such as pretraining, finetuning, domain adaptation and high-dimensional embedding spaces and quantify the gains in the example usecase of searches of heavy resonances decaying via an intermediate di-Higgs to four b-jets.

Significance

We demonstrate a finetuning workflow in the hierarchical setting of per-object representation and event-level inference within particle physics, quantifying the significant gains due to end-to-end optimization with respect to data efficiency and performance at fixed sample size. We also provide evidence of successful domain adaptation in a hierarchical setting of HEP foundation models finetuned on datasets other than the one they are pretrained with.

References

Paper: https://arxiv.org/abs/2401.13536
ML4Jets 2023: https://indico.cern.ch/event/1253794/contributions/5588562/
2023 ATLAS Flavour Tagging Workshop: https://indico.cern.ch/event/1311519/contributions/5582015/

Experiment context, if any CMS open data

Primary authors

Lukas Alexander Heinrich (Technische Universitat Munchen (DE)) Matthias Vigl (Technische Universitat Munchen (DE)) Nicole Michelle Hartman (TUM (DE))

Presentation materials