PHYSTAT-Flavour 2020

19 Oct 2020, 00:30 → 21 Oct 2020, 20:15 Europe/Zurich

Diego Tonelli (INFN Trieste, Italy), Hans Peter Dembinski (Max-Planck-Institute for Nuclear Physics, Heidelberg), Louis Lyons (Imperial College (GB)), Matthew William Kenzie (University of Warwick (GB)), Olaf Behnke (Deutsches Elektronen-Synchrotron (DE))

Zoom link

Meeting ID: 95191277014

Passcode: 318108

The virtual workshop deals with the

Statistical issues in modern flavour physics experiments

Topics to be included in the compact three afternoons workshop are                                 

• Principal inference techniques, also near physical boundaries
• Parameter estimation with using weighted events
• Systematic uncertainties

The workshop will be a mixture of invited talks from physicists and statisticians. One important goal is to collect a list of open issues in statistical treatments and procedures in flavour experiments that can be addressed in a future longer workshop with onsite participation.

The homepage of PHYSTAT with a list of all workshops is at https://espace.cern.ch/phystat.

Monday 19 October

Session 1

Conveners: Matthew William Kenzie (University of Warwick (GB)), Olaf Behnke (Deutsches Elektronen-Synchrotron (DE))

zoom_14h00-15h45.mp4

zoom_15h45-17h30.mp4

1 Introduction and Welcome

Speaker: Olaf Behnke (Deutsches Elektronen-Synchrotron (DE))

PHYSTATFL201019.pdf

2 Statistical issues in modern beauty and charm flavour physics experiments

We give an overview of common statistical issues discussed in the heavy flavour physics experiments LHCb and Belle. A focus will be put on limit setting in searches, the use of weighted events in inference (as computed with the sPlot technique), and the handling of systematic uncertainties.

Speaker: Hans Peter Dembinski (Max-Planck-Institute for Nuclear Physics, Heidelberg)

slides.pdf

14:35 Discussion time

3 Approaches to evaluating and reporting systematic uncertainties in flavour physics

The high luminosity and large cross sections enjoyed by LHC experiments means that statistical errors are minimal, and the rigorous treatment of systematic errors becomes very important - an area which lacks the "safety net" of chi squared and other goodness-of-fit measures. This entails including all uncertainties, estimating them properly, and not to inflating the error by including the results of consistency checks. This talk surveys recent papers by ATLAS, CMS and LHCb and examines how the collaborations handle the identification and estimation of systematic errors, the extent to which this is being done correctly, and what lessons can be learned.

Speaker: Roger Barlow (University of Huddersfield (GB))

fb.eps

talk.pdf

15:15 Discussion time

15:30 Virtual coffee break

Session 1

Conveners: Olaf Behnke (Deutsches Elektronen-Synchrotron (DE)), Matthew William Kenzie (University of Warwick (GB))

zoom_14h00-15h45.mp4

zoom_15h45-17h30.mp4

4 Testing a point null hypothesis versus a continuous alternative hypothesis

I will discuss aspects of the frequentist and Bayesian approaches to testing a point null hypothesis (say mu=0) versus a continuous alternative hypothesis (say mu>0). This test arises frequently in particle physics, where mu is the signal strength of a previously unobserved signal (within or beyond the Standard Model). The frequentist testing approach maps identically onto the frequentist theory of confidence intervals. Thus, as Feldman and Cousins eventually realized, the method advocated in their 1998 paper on confidence intervals maps identically onto the "classical" theory of likelihood ratio hypothesis tests in Kendall and Stuart (which in addition includes nuisance parameters). Meanwhile, the traditional Bayesian approach to hypothesis testing (due to Jeffreys) is completely separate from the Bayesian approach to credible intervals, with no corresponding mapping. Direct sensitivity to the prior pdf for mu, even in the asymptotic limit of large sample size, is a consequence, as is the Jeffreys-Lindley paradox (arXiv:1310.3791). My talk will draw on parts of my “Lectures on Statistics in Theory: Prelude to Statistics in Practice” (arXiv:1807.05996).

Speaker: Robert Cousins Jr (University of California Los Angeles (US))

cousins_phystat-flavour_oct2020.pdf

16:15 Discussion time

5 Interval Estimation, and the practice of Flavour Physics

Interval estimation is one of the most common types of inference practiced by experimentalists, and the flavour sector is not less interested than high-pt physics. In fact, the physics of flavour brings to the table some quite interesting problems, with complex multi-dimensional parameter spaces, non-linearities, and significant systematic effects. While much has been written on the theoretical principles of interval estimation, oftentimes the practitioner meets with issues for which practical guidance is not easy to find. Amongst typical hurdles are the correct handling of systematic uncertainties in frequentist intervals, and optimization of data selection for "best sensitivity" in the double-sided problem of discovery vs limits. I will give a discussion of those problems in the light of real-life experience from flavour physics measurements.

Speaker: Giovanni Punzi (Universita & INFN Pisa (IT))

phystat20_punzi.pdf

17:00 Discussion time

Tuesday 20 October

Session 2

Conveners: Diego Tonelli (INFN Trieste, Italy), Hans Peter Dembinski (Max-Planck-Institute for Nuclear Physics, Heidelberg)

zoom_14h00-15h45.mp4

zoom_15h45-17h15.mp4

zoom_18h00-19h00.mp4

6 Practical concerns in statistical combinations at ATLAS and CMS

The ATLAS and CMS collaborations have produced numerous results during the first two data-taking runs of the LHC, ranging from precision measurements of SM processes to searches for exotic phenomena and the discovery of the Higgs boson. These results make use of (often complex) statistical techniques, both for the publications and during the development and review of the data analysis/ In this talk, I will cover some of the common practices and tools used at both ATLAS and CMS to produce these results, and practical considerations for performing combinations across different channels or experiments.

Speaker: Nicholas Wardle (Imperial College (GB))

Practical_concerns_in_statistical_combinations_at_ATLAS_and_CMS.pdf

14:30 Disucssion time

7 Informative Goodness-of-Fit for Multivariate Distributions

In any experimental science, the knowledge available on a given phenomenon is formalized into a statistical model. The latter encapsulates our understanding of its nature, its properties as well as our uncertainties. Experimental measurements are then collected and statistical tests of hypothesis are used to answer the important question: is our model valid? As a result, a variety of tests for goodness-of-fit (GOF) have been proposed in the literature to study multivariate distributions. Despite their usefulness, classical GOF methods are somehow limited by their confirmatory nature. Specifically, when the respective null hypothesis is rejected, they do not allow us to identify the underlying causes which invalidate the model postulated by the scientists, nor they give any indication on how the latter can be improved to obtain a closer representation of the true data distribution. In simple words, they do not provide any insights on what went wrong. In this talk, I will introduce an informative goodness-of-fit (iGOF) approach to study multivariate distributions and which aims to address this issue directly. Specifically, when the null model is rejected, iGOF allows us to identify the underlying sources of mismodelling and naturally equip practitioners with additional insights on the underlying data distribution.

Speaker: Sara Algeri (University of Minnesota)

Phystat_Flavour.pdf

15:15 Disucssion time

15:30 Virtual coffee break

Session 2

Conveners: Louis Lyons (Imperial College (GB)), Dr Hans Peter Dembinski (Max-Planck-Institute for Nuclear Physics, Heidelberg)

zoom_14h00-15h45.mp4

zoom_15h45-17h15.mp4

zoom_18h00-19h00.mp4

8 Look-elsewhere effect in particle physics - knowns and unknowns

The LEE, also known as multiple comparisons problem and addressed by a trials factor, has become an essential part of any analyst's toolkit in the last decade. It is much less esoteric and much more pervasive that one could naively think and we'll walking through a series of first hand recounts of appearances it has made in the LHC. We'll both discuss known techniques to deal with the effect as well as potential caveats in hitherto unseen problems. While some potential problems are related to physics, the future is highly multi-dimensional and that brings us to present-day statistics issues that, interestingly, particle physicists may end up being reasonably equipped to tackle.

Speaker: André David (CERN)

20201020 PHYSTAT.pdf

16:15 Discussion time

17:00 Afternoon Break

9 Estimation, Accuracy and the Bootstrap

A brief introduction to boostrap estimates of accuracy, this talk does not assume familiarity with the topic. Bootstrap standard errors and confidence intervals are described using a small but genuine data set.

Speaker: Prof. Brad Efron (Stanford University)

talk2.pdf

18:30 Discussion time

Wednesday 21 October

Session 3

Conveners: Olaf Behnke (Deutsches Elektronen-Synchrotron (DE)), Matthew William Kenzie (University of Warwick (GB))

session3-part1.mp4

session3-part2.mp4

10 Using sWeights to disentangle signal and background

When an observable density is a superposition of signal and background PDFs that each factorise in a ''discriminant'' and a ''control'' variable, sWeights allow one to determine the signal density in the control variable using information from only the discriminant variable.
After reviewing the basics of the method and casting the formalism into the framework of orthogonal functions, the talk will address numerical aspects of the method and how to deal with weighted events or factorisation breaking.

Speaker: Michael Schmelling (Max-Planck-Gesellschaft (DE))

sweights_ms.pdf

14:30 Discussion time

11 Fitting weighted events

Fits of weighted events, for example to correct for acceptance effects or to statistically subtract background events using sWeights, have recently seen increasing use in the flavour physics community. This talk will discuss the determination of parameters and their uncertainties using weighted events, with particular focus on unbinned fits of weighted data.

Speaker: Christoph Michael Langenbruch (Rheinisch Westfaelische Tech. Hoch. (DE))

langenbruch.pdf

15:15 Discussion time

15:30 Virtual coffee break

Session 3

Conveners: Hans Peter Dembinski (Max-Planck-Institute for Nuclear Physics, Heidelberg), Louis Lyons (Imperial College (GB))

session3-part1.mp4

session3-part2.mp4

12 Goodness of Fit for unbinned data

Speaker: Francois Rene Le Diberder

PhyStat2020.pdf

16:15 Discussion time

13 Global Fits

Global fits are an indispensable tool in the search for New Physics (NP). On the one hand, they can provide interpretations of measurements that deviate from Standard Model (SM) predictions, and on the other hand, they allow using the wealth of experimental data for testing the viability of NP models. Being "global" means that these fits include hundreds of observables, whose theoretical predictions depend on a large number of nuisance parameters. Furthermore, the number of possible fit parameters in effective fields theories and explicit NP models can be huge. In this talk, I show how global fits are used in phenomenological analyses of flavour physics and other precision tests of the SM, I describe approximations that facilitate working with the large number of nuisance parameters, and I discuss issues and ambiguities related to the large number of possible fit parameters.

Speaker: Peter Stangl (University of Bern)

Stangl_20-10_phystat.pdf

17:00 Discussion time

14 Final remarks and closing of the workshop

Speakers: Diego Tonelli (INFN Trieste, Italy), Hans Peter Dembinski (Max-Planck-Institute for Nuclear Physics, Heidelberg), Louis Lyons (Imperial College (GB)), Matthew William Kenzie (University of Warwick (GB)), Olaf Behnke (Deutsches Elektronen-Synchrotron (DE))

PHYSTAT Summary-3.pdf