PHYSTAT-Anomalies

24 May 2022, 17:00 → 25 May 2022, 23:00 Europe/Zurich

Olaf Behnke (olaf.behnke@cern.ch) (Deutsches Elektronen-Synchrotron (DE)), Louis Lyons (l.lyons@physics.ox.ac.uk) (Imperial College (GB)), Ben Nachman (Lawrence Berkeley National Lab. (US)), Gregor Kasieczka (Hamburg University (DE)), Mikael Kuusela (Carnegie Mellon University (US))

A workshop on model independent searches, bringing together physicists and statisticians

With higher accelerator energies and beam intensities, searches for New Physics (NP) have been a very active area. While there is motivation for NP, so far model-driven searches have successfully excluded increasing volumes of parameter space, but not yielded evidence for new particles. This has led, over the last few years, to the development of model-independent searches. By now, they have become a useful complement to traditional approaches targeting some specific form of NP. In many of the new searches, Machine Learning has played an important role. The aim of this meeting is to compare and contrast the assumptions and performance on these approaches, and to see what can be learned from Goodness of Fit methodology. This meeting will bring together physicists who are active in this field, those who want to be involved in the future, and Statisticians, to discuss the relevant issues.

The meeting is on 24th and 25th May 2022, and will be remote. The PHYSTAT Seminar on 27th April by Mikael Kuusela (CMU) on "Model-Independent Detection of New Physics Signals Using Interpretable Semi-Supervised Classifier Tests"  is also part of the PHYSTAT-Anomalies meeting. See  https://indico.cern.ch/event/1148820/

 

The PHYSTAT series of Workshops started in 2000. They were the first meetings devoted solely to the statistical issues that occur in analyses in Particle Physics and neighbouring fields.The homepage of PHYSTAT with a list of all workshops and seminars is at https://espace.cern.ch/phystat . 

 

 

 

Tuesday 24 May

17:00 → 19:10 Session 1a. Chair: Mikael Kuusela

video_1a.mp4

17:00 Introduction

Speakers: Louis Lyons (Imperial College (GB)), Olaf Behnke (Deutsches Elektronen-Synchrotron (DE))

Intro_PHYSTAT_Anomalies_2022.pptx

17:10 Landscape of model independent searches

ABSTRACT:
This talk will briefly define and motivate anomaly detection at the Large Hadron Collider and will then give an overview of various method classifications based on the underlying physics goals and assumptions (and how these translate to statistical concepts). No one method will be able to cover all possibilities and it is essential to have a spectrum of techniques to achieve broad and deep model-independent sensitivity to physics beyond the Standard Model.

Speaker: Ben Nachman (Lawrence Berkeley National Lab. (US))

PhyStat2022AD.pdf

17:40 Questions

17:55 Learning New Physics from a machine

ABSTRACT:
Strategies to detect data departures from a given reference model, with no prior bias on the nature of the new physical model responsible for the discrepancy might play a vital role in experimental programs where, like at the LHC, increasingly rich experimental data are accompanied by an increasingly blurred theoretical guidance in their interpretation. I will describe one such strategy that employs neural networks, leveraging their virtues as flexible function approximants, but builds its foundations directly on the canonical likelihood-ratio approach to hypothesis testing. The algorithm compares observations with an auxiliary set of reference-distributed events, possibly obtained with a Monte Carlo event generator. It returns a p-value, which measures the compatibility of the reference model with the data. It also identifies the most discrepant phase-space region of the dataset, to be selected for further investigation. Imperfections due to mismodelling in the reference dataset can be taken into account straightforwardly as nuisance parameters. After illustrating the methodology, I will demonstrate its applicability to problems at a similar scale of complexity of realistic LHC analyses.

Speaker: Andrea Wulzer (Padova)

talk_wulzer.pdf

18:25 Questions

18:40 Discussion

18:55 Break

19:10 → 20:40 Session 1b. Chair Tom Junk

video_1b.mp4

19:10 Relation of model independent searches with Goodness of Fit, 2-sample tests, etc.

ABSTRACT:
I will review goodness of fit testing and two sample testing in the
context of trying to test for a new signal. My goal is to point to
results in the statistics literature that might be unfamiliar in the
physics community. Topics will include: optimal tests,
classifier-based tests, reproducing kernel Hilbert space tests, level set tests, bump tests and robustness.

Speakers: Larry Wasserman (Carnegie Mellon University), Larry Wasserman (Carnegie Mellon University)

Larry.pdf

19:40 Questions

19:55 Discussion

Speakers: Robert Cousins Jr (University of California Los Angeles (US)), Robert Cousins Jr (University of California Los Angeles (US))

cousins_phystat_anomalies_24may2022.pdf

Wednesday 25 May

17:00 → 19:00 Session 2a: Chair: Ann Lee

video_2a.mp4

17:00 Challenges of anomaly detection with LHC data

ABSTRACT:
In recent years, there have been many proposed methodologies for machine learning anomaly detection at the LHC, such as those reported in the LHC Olympics and Dark Machines community reports.  The first search using machine-learning anomaly detection was performed by ATLAS in the dijet final state, a fully data-driven analysis that uses the Classification Without Labels method and is complementary to the existing dedicated resonance searches. 
In this talk, I will use the experience gained with the ATLAS weakly supervised dijet analysis to discuss the general
challenges of doing anomaly detection with LHC data and methodologies to address them.

Speaker: Ines Ochoa (LIP Laboratorio de Instrumentacao e Fisica Experimental de Particulas (PT))

Phystat_iochoa_May2022.pdf

17:30 Questions

17:45 LHC Olympics

ABSTRACT:
We are at the beginning of a new era of data-driven, model-agnostic new physics searches at colliders that combine recent breakthroughs in anomaly detection and machine learning. This contribution will report on the LHC Olympics 2020, a community challenge accompanied by a set of simulated collider events. Participants in these Olympics have developed their methods using an R&D dataset and then tested them on black boxes: datasets with an unknown anomaly (or not). Methods made use of modern machine learning tools and were based on unsupervised learning (autoencoders, generative adversarial networks, normalizing flows), weakly supervised learning, and semi-supervised learning. We will review this challenge, including an overview of the competition, a description of methods deployed in the competition, lessons learned from the experience, and implications for data analyses with future datasets as well as future colliders.

Speaker: Gregor Kasieczka (Hamburg University)

DeepLearning_PHYSTAT_LHCO.pdf

18:15 Questions

18:30 Discussion

18:45 Break

19:00 → 20:30 Session 2b. Chair: David van Dyk

video_2b.mp4

19:00 Supervised, unsupervised and data-derived signal regions

ABSTRACT:
Data-driven methods are becoming increasingly popular and could give us new insights when searching for signals from new physics. On the other hand, theoretical models and supervised learning approaches should not be neglected.
In this talk we present and compare different ways of defining "signal regions" at the Large Hadron Collider that are of interest for a "goodness-of-fit" test. We compare the performance of 3 of these approaches and discuss a new way to define "signal regions".

Speaker: Sascha Caron (Nikhef National institute for subatomic physics (NL))

PhyStat_Anomaly2022.pdf

19:30 Questions

19:45 Discussion

Speaker: Richard Lockhart (SFU)

Lockhart2022PHYSTATAnomalies.pdf

20:15 Closing remarks

Speaker: Olaf Behnke (Deutsches Elektronen-Synchrotron (DE))

obehnke220525_phystatano.pdf