PHYSTAT informal review: CLs criterion for limit setting

by Alexander Lincoln Read (University of Oslo (NO)), Michael Evans (Toronto), Tom Junk (Fermi National Accelerator Lab. (US))

Wednesday 8 May 2024, 15:30 → 17:00 Europe/Zurich

This is the third PHYSTAT Informal Review event. In this new virtual format, a Tandem consisting of a physicist and a statistician will review a statistical method introduced by one of the parties or a general critical analysis topic from the Physicist's and Statistician's perspectives. The virtual events comprise: two 20+10 min. complementary presentations followed by ~30 minutes of general discussion. 

Today Tom Junk (Physicist, FNAL), Alex Read (Physicist, Oslo) and Mike Evans (Statistician, Toronto) will review the topic of   "CLs criterion for limit setting"

Agenda:

• 3.30 pm Opening 
• 3.30 pm Presentations by Alex Read & Tom Junk (20'+10')
• 4 pm Presentation Mike Evans (20'+10')
• 4.30 pm General Discussion and Closing (30')

 

Abstract:

We present an informal review on CL$_{\rm{s}}$, a technique designed for presenting the results of searches for new particles and interactions in high-energy physics experiments.  It is often called a "modified frequentist" method.  Its popularity in the high-energy physics community stems from its conservative relationship to a more traditional test of the test hypothesis which rejects the test hypothesis if its $p$ value is less than a pre-specified threshold, usually 0.05.  The CL$_{\rm{s}}$ method instead uses a ratio of two $p$ values, which lends it the desirable property of not being able to claim exclusions of test hypotheses for which the experiment did not have enough sensitivity to test.  We present the historical development, describe its properties, give examples of its use, and discuss alternatives.  Included in our review is a discussion of statistical evidence and its strength in a theory of statistical reasoning, comments on Birnbaum's work concerning statistical evidence, and Bayesian measurements of statistical evidence and its strength.  Concerns of  subjectivity of Bayesian methods are addressed, and how controlling error probabilities leads to considerable unification between Bayes and frequentism.

 

CLs_Physicist_Presentation.pdf

GMT20240508-133212_Recording_3686x2304.mp4

Mike_Evans_Slides.pdf