Publication of statistical models: hands-on workshop

8 Nov 2021, 14:00 → 12 Nov 2021, 19:00 Europe/Zurich

CERN (online only)

Sabine Kraml (LPSC Grenoble)

The statistical models used to derive the results of experimental analyses are of incredible scientific value and are essential information for analysis preservation and reuse. In arXiv:2109.04981, we made the scientific case for systematically publishing the full statistical models; we discussed the technical developments that make this practical, and illustrated by a variety of physics cases how detailed information on the statistical modelling can enhance the short- and long-term impact of experimental results

This workshop is intended as the first in a series to discuss in more detail practical issues for publishing statistical models and likelihoods, and work towards concrete solutions.

In this context note also the PHYSTAT workshop on systematics (Nov 1-3 + Nov 10) and in particular the talk by Kyle Cranmer on "A call to action: Honoring PHYSTAT's 20 year old agreement" at 6 pm CET on Nov 1st there, which will also in part set the stage for our workshop here.

Overall, apart from the first two days the workshop addresses a rather specialized audience, i.e. people want to who work on technical solutions for publishing and/or (re)using statistical models and likelihoods.

Slides and recordings of all sessions are available via the timetable.

Monday, 8 November

14:00 → 14:10 Worksop opening

Speaker: Sabine Kraml (LPSC Grenoble)

ws-opening.pdf

14:10 → 14:30 Introductory talk: statistical models and likelihoods

Speaker: Lukas Alexander Heinrich (CERN)

PubLhoodIntro.pdf

Recording (mp4)

14:40 → 17:10 Presentation of use cases

Convener: Sabine Kraml (LPSC Grenoble)

Recording: MA5 SModelS Protomodelling Flavour talks (mp4)

Recording: PDF EFT Higgs talks (mp4)

14:40 Parton Distribution Functions

Speaker: Juan Rojo (VU Amsterdam and Nikhef)

rojo-PDF-likelihoods.pdf

14:55 EFT fits

Speaker: Veronica Sanz Gonzalez (Universities of Valencia and Sussex)

publiclikelihoods.pdf

15:10 Higgs measurements

Speaker: Jonas Wittbrodt (Lund University)

wittbrodt.pdf

15:30 Break

16:00 Full and simplified likelihoods in MadAnalysis 5

Speaker: Jack Araz (IPPP - Durham University)

Full & simplified likelihoods in MadAnalysis5

MadAnalysis 5 - Home page for talks and tutorials

MadAnalysis 5 - Launchpad for questions and discussions

Public Analysis Database

16:15 Full and simplified likelihoods in SModelS

Speaker: Sabine Kraml (LPSC Grenoble)

smodels.pdf

16:30 Analysis combinations and proto-modelling

Speaker: Wolfgang Waltenberger (Austrian Academy of Sciences (AT))

08nov2021.pdf

16:45 Heavy flavor physics

Speaker: Florian Bernlochner (Uni Bonn)

Likelihood_Flavor_Talk.pdf

17:10 → 18:10 General discussion

Tuesday, 9 November

15:00 → 16:30 Hands on pyhf

Planned outline through pyhf tutorial material:

• Introduction to HistFactory
• Introduction to Workspaces
• Modifiers
• Workspace Manipulations
• Using HEPData
• Introduction to HistFactory Models with pyhf

Speaker: Matthew Feickert (Univ. Illinois at Urbana Champaign (US))

pyhf tutorial

pyhf tutorial GitHub repository

16:30 → 16:50 Break

16:50 → 18:20 Hands on Combine

• differences wrt HistFactory
• serialisation of Combine statistical models
• usage/extension of pyhf JSON format?
• ....

Speaker: Andrew Gilbert (Northwestern University (US))

Combine-LikelihoodWorkshop.pdf

Wednesday, 10 November

15:00 → 16:30 Summary talks from PHYSTAT workshop

We join the PHYSTAT workshop to follow their summary talks and discussion --> https://indico.cern.ch/event/1051224/timetable/

16:30 → 17:00 Break

17:00 → 18:30 Discussion on simplified likelihoods

Approaches, schemes, limitations; simplifying and pruning full statistical models (cf 3rd bullet point of Section 5 in arXiv:2109.04981)

Conveners: Andy Buckley (University of Glasgow (GB)), Nicholas Wardle (Imperial College (GB))

Recording (mp4)

17:00 Simplified likelihood frameworks

Speaker: Nicholas Wardle (Imperial College (GB))

paper ref.

simplifiedlikelihoods.pdf

SL code example

17:20 simplify

simplify-discussion.pdf

17:35 Fast approximate likelihoods of complex models

Speaker: Nicolas Berger (Centre National de la Recherche Scientifique (FR))

likelihoods_20211110.pdf

17:50 Discussion on simplified likelihoods

Thursday, 11 November

15:00 → 16:30 Free discussion and working session

This is a free session for those who want to discuss something. No fixed program or topic. Join main Zoom room and move to separate discussion room if needed.

15:30 → 17:00 Reinterpretation and likelihoods sessions of the LLP workshop

see https://indico.cern.ch/event/1042226/timetable/

to join:
URL: https://cern.zoom.us/j/66746428033?pwd=ZGYvZWo1dlExT3ZnamlRbzdlcHdoZz09
Meeting ID: 66746428033
Passcode: 96028080

LLP workshop timetable

16:30 → 17:00 Break

17:00 → 18:00 Joint session with LLP workshop

This is a joint session with the parallel "Long-lived Particle Community Workshop" where publication of likelihoods will be discussed. Please note the alternative indigo page and zoom details !

https://cern.zoom.us/j/69432885993?pwd=VWFEMnJFVFVxaW5JMnY2Rk53bzEvZz09
Passcode: 96028080

https://indico.cern.ch/event/1042226/timetable/#b-440156-re-interpretations-an

Convener: Louie Dartmoor Corpe (CERN)

Friday, 12 November

15:00 → 16:30 Machine learning likelihoods (and statistical models)

We intend to discuss the main ideas related to interpolating likelihoods and statistical models using (Deep) Neural Networks. The main topics and open questions/issues are:
- Bayesian vs Frequentist statistical approaches and their relations to the neural network representation of the Likelihood (e.g. combination of likelihoods and double counting of constraint terms vs priors, likelihood vs statistical model)
- Interpolation of full statistical models through NN vs other established approaches
- Regression vs density estimation (supervised vs unsupervised Likelihood learning)
- Practical implementations within experiments
- Practical implementations outside experiments (fitting groups)
- Examples

Conveners: Andrea Coccaro (INFN Genova (IT)), Riccardo Torre (INFN e Universita Genova (IT))

2021-11-12_Publication-Statistical-Models.pdf

GMT20211112-140831_Recording_1482x740.mp4

paper ref

16:30 → 17:00 break

17:00 → 18:30 Discussion on measurement-unfolding tools and combinations of LHC searches and measurements

Convener: Andy Buckley (University of Glasgow (GB))

Recording (mp4)

17:00 Intro

Speaker: Andy Buckley (University of Glasgow (GB))

2021-11 PubStatModels Unfolding.pdf

17:05 RooUnfold (esp IBU)

Speaker: Vincent Alexander Croft (Tufts University (US))

RooUnfoldSimplifiedWorkshop.pdf

17:10 TUnfold

Speaker: Stefan Schmitt (Deutsches Elektronen-Synchrotron (DE))

17:15 PyFBU

Speaker: Clement Helsens (CERN)

pyFBU_12-11-2021.pdf

17:20 TRExFitter

Speaker: Michele Pinamonti (Universita degli Studi di Udine (IT))

17:25 Convino

Speaker: Jan Kieseler (CERN)

17:30 Input from measurements with public stat models

17:35 Inputs from search, combination, and recasting