IML Machine Learning Working Group: LLMs for HEP
Tuesday 9 April 2024 -
15:00
Monday 8 April 2024
Tuesday 9 April 2024
15:00
News
-
Daniel Whiteson
(
University of California Irvine (US)
)
Anja Butter
(
Centre National de la Recherche Scientifique (FR)
)
Fabio Catalano
(
CERN
)
Lorenzo Moneta
(
CERN
)
Pietro Vischia
(
Universidad de Oviedo and Instituto de Ciencias y Tecnologías Espaciales de Asturias (ICTEA)
)
Stefano Carrazza
(
CERN
)
Julian Garcia Pardinas
(
CERN
)
News
Daniel Whiteson
(
University of California Irvine (US)
)
Anja Butter
(
Centre National de la Recherche Scientifique (FR)
)
Fabio Catalano
(
CERN
)
Lorenzo Moneta
(
CERN
)
Pietro Vischia
(
Universidad de Oviedo and Instituto de Ciencias y Tecnologías Espaciales de Asturias (ICTEA)
)
Stefano Carrazza
(
CERN
)
Julian Garcia Pardinas
(
CERN
)
15:00 - 15:05
Room: 500/1-001 - Main Auditorium
15:05
AccGPT – A CERN Chatbot for Internal Knowledge Retrieval
-
Florian Rehm
(
CERN
)
AccGPT – A CERN Chatbot for Internal Knowledge Retrieval
Florian Rehm
(
CERN
)
15:05 - 15:20
Room: 500/1-001 - Main Auditorium
15:20
Discussion
Discussion
15:20 - 15:25
Room: 500/1-001 - Main Auditorium
15:25
Learning the language of QCD jets with transformers
-
Michael Kramer
(
Rheinisch Westfaelische Tech. Hoch. (DE)
)
Alexander Mück
(
RWTH Aachen University
)
Learning the language of QCD jets with transformers
Michael Kramer
(
Rheinisch Westfaelische Tech. Hoch. (DE)
)
Alexander Mück
(
RWTH Aachen University
)
15:25 - 15:40
Room: 500/1-001 - Main Auditorium
15:40
Discussion
Discussion
15:40 - 15:45
Room: 500/1-001 - Main Auditorium
15:45
OmniJet-α: The first cross-task foundation model for particle physics
-
Joschka Birk
(
Albert Ludwigs Universitaet Freiburg (DE)
)
Gregor Kasieczka
(
Hamburg University (DE)
)
Anna Maria Cecilia Hallin
(
University of Hamburg
)
OmniJet-α: The first cross-task foundation model for particle physics
Joschka Birk
(
Albert Ludwigs Universitaet Freiburg (DE)
)
Gregor Kasieczka
(
Hamburg University (DE)
)
Anna Maria Cecilia Hallin
(
University of Hamburg
)
15:45 - 16:00
Room: 500/1-001 - Main Auditorium
Foundation models are multi-dataset and multi-task machine learning methods that once pre-trained can be fine-tuned for a large variety of downstream applications. The successful development of such general-purpose models for physics data would be a major breakthrough as they could improve the achievable physics performance while at the same time drastically reduce the required amount of training time and data. We report significant progress on this challenge on several fronts. First, a comprehensive set of evaluation methods is introduced to judge the quality of an encoding from physics data into a representation suitable for the autoregressive generation of particle jets with transformer architecture (the common backbone of foundation models). These measures motivate the choice of a higher-fidelity tokenization compared to previous works. Finally, we demonstrate transfer learning between an unsupervised problem (jet generation) and a classic supervised task (jet tagging) with our new OmniJet-α model. This is the first successful transfer between two different and actively studied classes of tasks and constitutes a major step in the building of foundation models for particle physics.
16:00
Discussion
Discussion
16:00 - 16:05
Room: 500/1-001 - Main Auditorium
16:05
ChATLAS: developing an AI assistant for the ATLAS collaboration
-
Daniel Thomas Murnane
(
Lawrence Berkeley National Lab. (US)
)
ChATLAS: developing an AI assistant for the ATLAS collaboration
Daniel Thomas Murnane
(
Lawrence Berkeley National Lab. (US)
)
16:05 - 16:20
Room: 500/1-001 - Main Auditorium
16:20
Discussion
Discussion
16:20 - 16:25
Room: 500/1-001 - Main Auditorium