PHYSTAT Seminar: Double Descent, Overparametrization and Scaling Laws in Particle Physics Data

by Matthias Vigl (Technische Universitat Munchen (DE))

Wednesday 7 Oct 2026, 16:00 → 17:15 Europe/Zurich

This 45'+30' seminar is about the paper https://arxiv.org/abs/2509.01397

 

Abstract: Much of the recent progress in machine learning has come from scaling compute, with frontier LLMs now exceeding 10^26 training FLOPs and trillions of parameters, orders of magnitude beyond state-of-the-art HEP models. Empirical scaling laws make performance predictable as a function of compute and guide its allocation between model size and training data. We study how this scaling paradigm translates to particle physics across a range of datasets and tasks. We first consider fixed training datasets, where increasing model size beyond the interpolation threshold can continue to improve generalization through double descent. These gains are eventually limited by the available data, motivating the joint scaling of model size and training data with compute. We then derive compute-optimal scaling laws for transformer-based jet taggers trained on up to billions of simulated jets and study how these scaling relations depend on the training hyperparameters and input representation. The observed scaling behavior suggests that substantially larger compute budgets could be productively invested, particularly if the field moves toward more general-purpose foundation models.

---

GMT20261007-140346_Recording_1832x982.mp4

PHYSTAT_Scaling.pdf