IML Machine Learning Working Group: LLMs for HEP

Europe/Zurich
500/1-001 - Main Auditorium (CERN)

500/1-001 - Main Auditorium

CERN

400
Show room on map
Description

Topic: LLMs for HEP

Videoconference
IML Machine Learning Working Group
Zoom Meeting ID
96543252431
Host
Simon Akar
Alternative hosts
Riccardo Torre, Fabio Catalano
Passcode
09010263
Useful links
Join via phone
Zoom URL
    • 3:00 PM 3:05 PM
      News 5m
      Speakers: Anja Butter (Centre National de la Recherche Scientifique (FR)), Daniel Whiteson (University of California Irvine (US)), Fabio Catalano (CERN), Julian Garcia Pardinas (CERN), Lorenzo Moneta (CERN), Dr Pietro Vischia (Universidad de Oviedo and Instituto de Ciencias y Tecnologías Espaciales de Asturias (ICTEA)), Stefano Carrazza (CERN)
    • 3:05 PM 3:20 PM
      AccGPT – A CERN Chatbot for Internal Knowledge Retrieval 15m
      Speaker: Dr Florian Rehm (CERN)
    • 3:20 PM 3:25 PM
      Discussion 5m
    • 3:25 PM 3:40 PM
      Learning the language of QCD jets with transformers 15m
      Speakers: Dr Alexander Mück (RWTH Aachen University), Michael Kramer (Rheinisch Westfaelische Tech. Hoch. (DE))
    • 3:40 PM 3:45 PM
      Discussion 5m
    • 3:45 PM 4:00 PM
      OmniJet-α: The first cross-task foundation model for particle physics 15m

      Foundation models are multi-dataset and multi-task machine learning methods that once pre-trained can be fine-tuned for a large variety of downstream applications. The successful development of such general-purpose models for physics data would be a major breakthrough as they could improve the achievable physics performance while at the same time drastically reduce the required amount of training time and data.
      We report significant progress on this challenge on several fronts. First, a comprehensive set of evaluation methods is introduced to judge the quality of an encoding from physics data into a representation suitable for the autoregressive generation of particle jets with transformer architecture (the common backbone of foundation models). These measures motivate the choice of a higher-fidelity tokenization compared to previous works. Finally, we demonstrate transfer learning between an unsupervised problem (jet generation) and a classic supervised task (jet tagging) with our new OmniJet-α model. This is the first successful transfer between two different and actively studied classes of tasks and constitutes a major step in the building of foundation models for particle physics.

      Speakers: Anna Maria Cecilia Hallin (University of Hamburg), Gregor Kasieczka (Hamburg University (DE)), Joschka Birk (Albert Ludwigs Universitaet Freiburg (DE))
    • 4:00 PM 4:05 PM
      Discussion 5m
    • 4:05 PM 4:20 PM
      ChATLAS: developing an AI assistant for the ATLAS collaboration 15m
      Speaker: Daniel Thomas Murnane (Lawrence Berkeley National Lab. (US))
    • 4:20 PM 4:25 PM
      Discussion 5m