Reinforcement Learning

by Matteo Bunino (CERN)

Tuesday 25 Aug 2026, 09:00 → 16:00 Europe/Zurich

593/R-010 - Salle 11 (CERN)

Abstract

Reinforcement Learning (RL) is behind much of the recent progress in AI, from agents that master games to the reasoning abilities of modern language models. This lecture follows a single line from the foundations of RL to the algorithms used to train frontier models today.

We start with the fundamentals: agents interacting with an environment, Markov Decision Processes, value functions, and learning from experience through Monte Carlo and Temporal Difference methods. Function approximation then replaces lookup tables with neural networks, bringing us to Deep Q-Networks and the instabilities they introduce.

The second part turns to methods that optimise the policy directly. From REINFORCE we see why variance is prohibitive, how a baseline fixes it, and how learning that baseline gives actor-critic methods. The fact that policy gradient data cannot be reused motivates trust regions and Proximal Policy Optimization, which in turn opens the door to language models: Reinforcement Learning from Human Feedback, verifiable rewards for mathematics and code, and Group Relative Policy Optimization, the algorithm behind DeepSeek-R1.

We close on what goes wrong in practice. Reward hacking, entropy collapse, and catastrophic forgetting are central obstacles rather than corner cases, and each has a diagnosis worth knowing. The emphasis throughout is on why an algorithm is built the way it is rather than on how to implement it.

A practice session follows, in which participants fine-tune a small language model with GRPO, watch it lose general ability as it improves at its task, and implement the countermeasure. Familiarity with Python and neural networks is assumed; no prior RL knowledge is required.

Lecturer Bio 

Matteo Bunino earned a double MSc in Data Science and Computer Engineering from Politecnico di Torino (Italy) and EURECOM (France). He completed his thesis at Huawei's Munich Research Center, applying reinforcement learning to the analysis of dynamically evasive malware.

After university, Matteo joined CERN, where he worked on interTwin, a European project developing a digital twin engine for science, and contributed to CERN openlab. During this time he focused on the development of itwinai, a framework for advanced MLOps on cloud and HPC.

Matteo is currently part of the Kubernetes platform team in the CERN IT department, where he works on the ODISSEE project, whose goal is to create the digital twin of the LHCb data centre.