pyhf Users and Developers Workshop 2023

4 Dec 2023, 09:00 → 8 Dec 2023, 17:00 Europe/Zurich

53/R-044 (CERN)

Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Lukas Alexander Heinrich (Technische Universitat Munchen (DE)), Matthew Feickert (University of Wisconsin Madison (US))

What is this workshop?

This is the inaugural pyhf workshop for users and developers across all of physics (following the 2023 Belle II pyhf workshop). The goals of the workshop are twofold:

• Gather the pyhf user community together to learn from their experience and better understand what changes will improve things for the most number of people.
  
  We expect to have contributions from:
  • Scikit-HEP
  • IRIS-HEP
  • ATLAS
  • CMS (sorry this conflicts with CMS week)
  • Belle II
  • MicroBooNE
  • SModelS
  • MadAnalysis 5
  • and more
• Educate users who are interested in becoming contributors to the pyhf codebase and onboard those interested in becoming project maintainers. This part of the workshop will take the form of "sprints" (inspired from the SciPy conference sprints) where contributors can work on making contributions with the development team. Despite their name, the sprints are meant to be relaxed reserved times for development, and aren't meant to be chaotic or exhausting.

Workshop video recordings

All talks from the workshop are posted to a playlist on the IRIS-HEP YouTube channel and are professionaly captioned in English. Links to the recordings are also added to each talk's Indico agenda page.

Call for Abstracts

The user section of the workshop will include talks that will be a mix of invited and solicited abstract proposals. A call for abstracts will be announced in October 2023.

Should I come to this workshop?

If you use pyhf in any way for your work, or if you would like to learn more about pyhf, then yes! You and your ideas are very welcome.

Thanks to support from NumFOCUS there will be limited travel financial aid that can be applied for. See the registration for more details.

Connecting remotely with Zoom

While the workshop will primarily be in-person there will be a Zoom connection available for those who are unable to travel to CERN.

User Community Project Collection

Do you have a project that uses pyhf? If so, and please open a GitHub Issue to add it to the pyhf user project list: github.com/pyhf/user-projects 

Rooms and Map

 

Workshop photos

• ATLAS cavern tour

Monday 4 December

10:00 → 11:40 Users Section: pyhf users introduction

Conveners: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Matthew Feickert (University of Wisconsin Madison (US))

10:00 Workshop welcome

Speaker: Matthew Feickert (University of Wisconsin Madison (US))

Community Notes

pyhf-workshop-introduction_2023-12-04.pdf

YouTube Recording

10:10 pyhf Overview and Basic Tutorial

This will warm up new users to what pyhf can do. Experienced users of pyhf will be able to sit back and gain a nice refresher.

Speaker: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US))

pyhf docs

pyhf GitHub

pyhf tutorial

YouTube Recording

11:00 Constructing and steering pyhf models with cabinetry

The cabinetry library provides interfaces and functionality for both the creation and use of statistical models together with pyhf. Models can be built from instructions provided in a declarative configuration. A high-level inference API and visualization utilities help study and disseminate fit results.

This talk will provide an overview of cabinetry and also highlight recent additions and ongoing developments.

Speaker: Alexander Held (University of Wisconsin Madison (US))

20231204_cabinetry_pyhf_workshop.pdf

repository with all material

YouTube Recording

11:40 → 12:10 Users Section: Feedback from LHC experiments

Convener: Matthew Feickert (University of Wisconsin Madison (US))

11:40 Modernizing the ATLAS Statistical Analysis: Implementing HistFitter Strategies with pyhf in Supersymmetry search

This contribution signifies a shift in ATLAS statistical data analysis by implementing traditional fit strategies utilizing the pyhf library, alongside the cabinetry library. Leveraging a toy Supersymmetry search analysis, three fit strategies inspired by the HistFitter framework are implemented. The "background-only fit," "model-dependent signal fit," and "model-independent signal fit" strategies show the adaptability of pyhf, liberating the analysis from dependence on traditional ROOT-based tools. In addition to enhancing clarity regarding the statistical model itself, this implementation signifies a broader shift towards contemporary standards in data analytics.

Speaker: Vangelis Kourlitis (Technische Universitat Munchen (DE))

pyhf2023 ATLAS SUSY Fits

YouTube Recording

14:00 → 16:00 Users Section: Feedback from LHC experiments

Convener: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US))

14:00 abcd_pyhf: Likelihood-based ABCD method for background estimation and hypothesis testing with pyhf

The ABCD method is a common background estimation method used by many physics searches in particle collider experiments and involves defining four regions based on two uncorrelated observables. The regions are defined such that there is a search region (where most signal events are expected to be) and three control regions. A likelihood-based version of the ABCD method, also referred to as the "modified ABCD method", can be used even when there may be significant contamination of the control regions by signal events. abcd_pyhf is a standalone implementation of this method utilizing pyhf. This implementation does not make any assumptions about the underlying analysis and can thus be used or adapted in any analysis using the ABCD method. This lightning talk will summarize the abcd_pyhf project and its current status.

Speaker: Mason Proffitt (University of Washington (US))

Binder (public)

GitHub

SSL BinderHub

YouTube Recording

14:30 The HEP Statistics Serialization Standard (HS³)

The HEP Statistics Serialization Standard is a new format in which to
write, store, exchange and archive statistical models. It is based on
JSON and easily readable and writable for both machines and humans. It
is fully interoperable with ROOT workspaces, the current de-facto
standard in the experimental HEP community, and has a concept
implementation in Julia / BAT.jl and intends to be interoperable with
pyhf, zfit and any other large-scale, general-purpose statistical
modeling library used in HEP. It supports binned and unbinned datasets
and models, as well as a growing library of distributions and
auxiliary functions. It is bidirecionally convertible to ROOT
workspaces and pyhf JSON files.

While the standard is already in-use and implementations covering most
currently defined features exist, the development of the standard is
ongoing, and community feedback of any type is highly welcome.

Speakers: Dr Carsten Burgard (Technische Universitaet Dortmund (DE)), Robin Pelkner (Technische Universitaet Dortmund (DE))

HS3

YouTube Recording

15:00 HEPExplorer, an interactive visualization tool for pyhf HistFactory workspaces.

HEPExplorer is a web based viewer for high energy particle physics that allows users to view various types of plots from data formatted as HistFactory workspaces. This simple tool provides an easy-to-use and convenient way to generate plots, perform fits, and to investigate the impact of various parameters on the model performance using HistFactory workspaces.

HF Explorer is intended to be a tool for high energy physicists working with HistFactory data. It allows for a quick and easy way to view results and to perform some very basic exploration of model parameters. The benefit of this web based approach is that this tool can be used quickly without any software installation, and it provides a means of easily sharing results via url. It is intended to provide convenience and simplicity over installing and running the pyhf / cabinetry tools locally via command line. This type of tool is intended to encourage collaboration by encouraging data sharing and interaction.

Benefits:
Easy to use workspace visualization.
No software installation required.
Runs on any internet connected device.
Easily share results via url.

Features:
Display histograms.
Display pull plots.
Display and interact with model parameters.
Perform fits.
Lock selected model parameters during the fit process.
Sort model parameters by name or by importance / impact.
Compare channel histograms side by side.

The application is available at: https://hepexplorer.net. The code is Open Source (MIT License) and available from: https://github.com/UW-Madison-DSI/HFExplorer

We look forward to user feedback and to an opportunity to make this simple viewing tool better serve the HEP community.

Speaker: Abe Megahed (Data Science Institute, University of Wisconsin-Madison)

HF Explorer GitHub Repository

HF Explorer.pdf

YouTube Recording

15:30 WorkspaceExplorer

pyhf, in combination with cabinetry, has been used successfully in a statistical combination of searches for Beyond-Standard-Model particles by the ATLAS Combination. Since for the individual searches various different frameworks were used to perform the statistical analysis, an essential part of the combination effort consisted of validating the statistical workspaces after they had been converted into the pyhf format for full likelihoods. As part of the validation process, I developed a web-based interface to easily and quickly visualise workspace contents and perform fits. This so-called WorkspaceExplorer is available at workspaceexplorer.app.cern.ch. In addition to validation, it can for example also serve educational purposes or aid in exploration of unfamiliar statistical models. In this talk, I will present available features and discuss possible use cases for the tool as well as the potential for further developments.

Speaker: Volker Andreas Austrup (University of Manchester (GB))

pyhfworkshop_workspaceexplorer.pdf

YouTube Recording

Tuesday 5 December

10:00 → 11:30 Users Section: Feedback from Theory and HEPData

Conveners: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Matthew Feickert (University of Wisconsin Madison (US))

10:00 HEPData and pyhf

I will give an overview of the HEPData project (hepdata.net), a repository for publication-related data from experimental particle physics. I will describe current support for hosting statistical models in the HistFactory JSON format used by pyhf, as well as links with analysis frameworks like Rivet and MadAnalysis 5. The HEPData software is open source under the @HEPData organization on GitHub and external contributions are welcome.

Speaker: Graeme Watt

watt_hepdata_dec2023.pdf

YouTube Recording

10:30 Signal region combination in CheckMATE

I will present new features of CheckMATE, in particular implementation of multibin fits in a number of ATLAS and CMS searches.
I will discuss some examples of implemented analysis, show an application of the method to electroweakino scenarios and discuss notable improvements in exclusion range due to CMS multijet search.

Speaker: Iñaki Lara Perez

pyhfUsers_Ilara.pdf

YouTube Recording

11:00 Staying on Top of likelihood analyses: Using pyhf in the context of global SMEFT analyses with SFitter

The SFitter analysis framework has been used for many global analyses, making use of a comprehensive treatment of uncertainties and their correlations to provide constraints on the Standard Model Effective Field Theory (SMEFT). Due to the nature of global analyses, this requires the implementation of a large number of different experimental measurements. The publication of likelihoods by the experimental collaborations, along with the use of pyhf, now allows for a new approach to the implementation of this data into SFitter. I will give an overview of the SFitter framework, highlighting in particular how it takes uncertainties and correlations between measurements into account and demonstrating the use of pyhf for the implementation of likelihoods in this context.

Speaker: Nikita Schmal

pyhf_SMEFT_schmal.pdf

YouTube Recording

14:00 → 17:20 Users Section: Feedback from Flavor and Neutrino Physics

Convener: Matthew Feickert (University of Wisconsin Madison (US))

14:00 Searching for BSM physics with the MicroBooNE detector

The MicroBooNE detector is a liquid argon time projection chamber located on the Fermilab campus. It has excellent calorimetric and spatial reconstruction capabilities. Moreover, MicroBooNE is exposed to two neutrino beams, the Booster Neutrino Beam (on-axis) and the Neutrinos at the Main Injector beam (off-axis). These outstanding features make MicroBooNE an ideal experiment to search for beyond the Standard Model (BSM) signatures, such as eV-scale sterile neutrinos, heavy neutral leptons (HNLs), millicharged particles, and light dark matter. Probing the existence of any of these BSM candidates involves using statistical methods such as hypothesis testing and limit setting. Pyhf offers a stand-alone framework that implements many of the statistical methods used in high energy physics. In this talk, we present the experience of MicroBooNE with pyhf in the context of HNL and light-dark matter searches. We highlight the advantages that pyhf offered to both analyses. In addition, we discuss a set of desired features that could be implemented in the framework.

Speaker: Luis Alberto Mora Lepin (University of Manchester (GB))

pyhf_workshop_microboone-4.pdf

YouTube Recording

14:30 Introduction to the Belle II collaboration and feedback from its analyzers

The Belle II experiment, located at the SuperKEKB e$^{+}$e$^{−}$ collider at KEK (Japan), precisely measures the Standard Model parameters analyzing various flavor physics processes to search for new physics beyond the Standard Model. It has collected a data set with an integrated luminosity of 428 fb$^{−1}$ and a peak instantaneous luminosity of 4.7 × 10$^{34}$ cm$^{−2}$ s$^{−1}$. The physic program of Belle II covers a wide scope, including B, charm, τ , quarkonium physics, electroweak precision measurements and dark sector searches. This talk will give an introduction to the Belle II detector, its collected data and the collaboration structure. It will give an overview of the ongoing physics analyses, with a focus on the underlying fitting frameworks. In addition, it contains feedback for the pyhf-developers collected within the Belle II collaboration.

Speaker: Fabian Becherer (DESY)

belle-2-introduction.pdf

YouTube Recording

15:00 Feedback from SModelS

I will discuss the use we make of published statistical models and patchsets in SModelS. I'll cover benefits as well as difficulties encountered, and wishes for future developments.

Speaker: Sabine Kraml (LPSC Grenoble)

smodels.pdf

YouTube Recording

15:30 Coffee/Tea Break

16:00 Usage of full likelihoods in reinterpretation studies

In this talk, we will provide an overview of the usage of full likelihoods through pyhf package within LHC reinterpretation studies and software such as MadAnalysis 5, SModelS and spey. We will also provide a summary of future directions involving pyhf package, such as the combination of statistical models, simplified likelihoods via ML approaches and converting full statistical models to a simplified likelihood framework.

Speaker: Jack Y. Araz (Jefferson Lab)

Spey documentation

Usage of full likelihoods in reinterpretation studies

YouTube Recording

16:30 A decade of HistFactory

I'll share a few thoughts on the past, present, and future of the HistFactory specification and pyhf.

Speaker: Kyle Stuart Cranmer (University of Wisconsin Madison (US))

pyhf-workshop-2023.pdf

YouTube Recording

17:00 Introducing the FAIROS-HEP Research Coordination Network

The US National Science Foundation has funded a 3-year "Research Coordination Network" called FAIROS-HEP. FAIROS-HEP aims to foster the adoption of practices and cyberinfrastructure to enable reuse and reinterpretation of high energy physics (HEP) datasets. The network has funds to support international workshops and to contribute directly to cyberinfrastructure components such as INSPIRE, HEPData, etc. A specific aim of FAIROS-HEP is to aid in developing a framework for EFT global fits as conceptualized by the community through meetings such as the LHC EFT WG and to aid in other BSM reinterpretation activities. The purpose of this talk is just to announce the project and the availability of funds for participants to attend dedicated workshops and targeted contributions to cyberinfrastructure components.

Speaker: Kyle Stuart Cranmer (University of Wisconsin Madison (US))

2023-12-05-FAIROS-pyhf-workshop.pdf

YouTube Recording

18:00 → 19:00 Social: ATLAS Cavern Underground Tour

Please fill out the survey if you would like to attend: https://indico.cern.ch/event/1294577/surveys/4882

Conveners: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Lukas Alexander Heinrich (Technische Universitat Munchen (DE)), Matthew Feickert (University of Wisconsin Madison (US)), Steven Goldfarb (University of Melbourne (AU))

19:30 → 21:30 Social: Users Social Dinner

Please fill out the survey if you would like to attend: https://indico.cern.ch/event/1294577/surveys/4882

Conveners: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Matthew Feickert (University of Wisconsin Madison (US))

Wednesday 6 December

10:00 → 12:00 Users Section: Discussion of Common Problems

Conveners: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Matthew Feickert (University of Wisconsin Madison (US))

10:00 Discussion of Common Problems

Speakers: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Matthew Feickert (University of Wisconsin Madison (US))

Workshop Community Notes Google Doc

14:00 → 16:00 Users Section: Discussions of New Ideas

Conveners: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Matthew Feickert (University of Wisconsin Madison (US))

14:00 Discussion of New Ideas

Speakers: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Lukas Alexander Heinrich (Technische Universitat Munchen (DE)), Matthew Feickert (University of Wisconsin Madison (US))

Workshop Community Notes Google Doc

16:00 → 16:10 Group Photo

Speakers: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Matthew Feickert (University of Wisconsin Madison (US))

16CFA766-E027-4763-AA8A-29F1F8A0A93D.jpg

712C5561-C180-4AF7-82C0-423E615A8DF7.jpg

9A389407-5628-4137-91DE-981CF96E5A87.jpg

F144436D-13B7-43D5-9AE7-8642BDE4C761.jpg

F22EFDB0-2E01-48FE-B52F-AB2A09627C75.jpg

16:10 → 16:40 Users Section: Users Sessions Closeout

Convener: Matthew Feickert (University of Wisconsin Madison (US))

Thursday 7 December

10:00 → 10:30 dilax: Differentiable Binned Likelihoods in JAX

dilax is a software package for statistical inference with binned likelihoods. It focusses on three key concepts: performance, differentiability, and object-oriented statistical model building. Thus, dilax is build upon the shoulders of a deep learning giant: JAX - a popular autodifferentiation Python framework. By making every component in dilax a PyTree, each component can be jit-compiled (jax.jit), vectorized (jax.vmap) and differentiated (jax.grad). This does not only fulfil all key concepts, but also enables novel computational concepts, such as running thousands of fits simultaneously on a GPU.
We present the key concepts of dilax, show its features, and discuss performance benchmarks with toy datasets.

Speaker: Manfred Peter Fackeldey (RWTH Aachen University (DE))

dilax_pyhf_workshop.pdf

YouTube Recording

10:30 → 11:00 Developers Section: Development Onboarding

Conveners: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Matthew Feickert (University of Wisconsin Madison (US))

11:00 → 12:00 Developers Section: Birds of a Feather

Conveners: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Lukas Alexander Heinrich (Technische Universitat Munchen (DE)), Matthew Feickert (University of Wisconsin Madison (US))

11:00 BoF Overview

Speakers: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Lukas Alexander Heinrich (Technische Universitat Munchen (DE)), Matthew Feickert (University of Wisconsin Madison (US))

11:15 BoF Breakouts (TBD)

Speakers: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Lukas Alexander Heinrich (Technische Universitat Munchen (DE)), Matthew Feickert (University of Wisconsin Madison (US))

14:00 → 16:30 Developers Section: Sprints

Conveners: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Lukas Alexander Heinrich (Technische Universitat Munchen (DE)), Matthew Feickert (University of Wisconsin Madison (US))

17:00 → 18:30 Social: Sprints Social Dinner

Please fill out the survey if you would like to attend: https://indico.cern.ch/event/1294577/surveys/4882

Convener: Matthew Feickert (University of Wisconsin Madison (US))

Friday 8 December

10:00 → 16:00 Developers Section: Sprints

Conveners: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Lukas Alexander Heinrich (Technische Universitat Munchen (DE)), Matthew Feickert (University of Wisconsin Madison (US))

16:00 → 16:20 Developers Section: Developers Closeout

Conveners: Dr Giordon Holtsberg Stark (University of California,Santa Cruz (US)), Lukas Alexander Heinrich (Technische Universitat Munchen (DE)), Matthew Feickert (University of Wisconsin Madison (US))