Statistical Analysis Software Meeting

→ Europe/Zurich
Jonas Rembser (CERN)
Zoom Meeting ID
62468501683
Host
Jonas Rembser
Useful links
Join via phone
Zoom URL

Statistical Analysis Software Meeting — Minutes

This is an AI-generated summary from the meeting transcript.

Topic: HistFactory V2 — Gaussian Process Regression for Template Interpolation Speaker: Jay Sandesara Chair: Jonas Rembser

Summary

Jay presented HistFactory V2, a proposed replacement for the factorised parametric interpolation currently used in HistFactory V1. The new approach uses Gaussian Process (GP) regression to interpolate between simulation anchor points in the high-dimensional nuisance parameter space. A first prototype implementation has been built in a fork of pyhf via a new GP_histosys-style additive modifier, and toy and pyhf-based examples reproduce HistFactory V1 exactly when only on-axis ±1σ templates are provided, while correctly fitting off-axis anchor points when additional templates are supplied.

Presentation Highlights

Motivation and limitations of HistFactory V1

  • HistFactory V1 assumes that the impact of independent nuisance parameters on the likelihood factorises and uses ad hoc parametric forms (e.g. polynomial interpolation through nominal, +1σ, −1σ templates).
  • These assumptions can break down even when nuisance parameters are independent, and they are particularly weak for theoretical uncertainties, where parameters can be correlated.
  • The model may be insufficiently expressive in cases with non-smooth or non-factorising dependencies, potentially obscuring or mimicking new-physics effects.

The HistFactory V2 proposal

  • Model the template-deviation term δ(α) using Gaussian Process regression over the full high-dimensional α space, rather than dimension by dimension.
  • The GP takes as input the anchor points, the corresponding simulated histogram templates, a prior mean (the HistFactory V1 prediction) and a prior covariance kernel; it returns a posterior mean and posterior covariance.
  • The posterior mean is used as the central prediction; the posterior covariance provides an interpolation/extrapolation uncertainty that could be promoted to an additional constrained nuisance parameter.
  • Because the prior is the HistFactory V1 prediction, the GP reproduces HistFactory V1 exactly when no additional information is supplied and only adds corrections where extra anchor points (or, in principle, gradient information) are available.
  • The approach was originally proposed by Lukas Heinrich; Jay extended it and built a first end-to-end implementation.

Additional features

  • Off-axis anchor points (e.g. simulations at simultaneous variations of multiple nuisance parameters) can be ingested directly without changing the rest of the model.
  • Bin-wise Monte Carlo statistical uncertainty can be encoded as noise on the anchor points, giving the GP additional flexibility at points with low MC statistics.
  • Gradient information at anchor points can in principle be used as an additional inductive bias, although this is not yet studied in practice.

pyhf prototype

  • Implemented as a new additive modifier GP_histosys (additive to match pyhf's code 4p interpolation exactly when used as a prior).
  • The workspace spec lists anchor nodes, the corresponding bin-wise templates and labels for each nuisance parameter dimension; prior length scale and noise are configurable.
  • Toy 2-bin / 2D examples (notebooks linked from Jay's slides):
    • With only on-axis ±1σ anchor points: GP_histosys reproduces histosys exactly along axis-aligned and diagonal slices in α₁–α₂.
    • With additional off-axis anchors at (+1,+1) and (−1,−1): histosys mispredicts the diagonal, while GP_histosys fits the off-axis points and gives correlations consistent with the truth.

Outlook

  • Plan to publish a paper on HistFactory V2.
  • Aim to provide implementations both in pyhf and in ROOT, and to establish a clean, possibly HS3-compatible serialisation that, for example, references nuisance parameter names rather than long explicit α-vectors.
  • Even without using the posterior covariance as a nuisance parameter, it can serve as a diagnostic for under-constrained interpolation regions.

Discussion

Dimensionality reduction (Carsten Burgard)

Carsten asked whether the procedure could be combined with a PCA-style rotation of the nuisance parameter space, so that the leading directions are identified from the distribution of anchor points and only those are kept as constrained nuisance parameters — analogous to what ATLAS/CMS combined-performance groups already do for, e.g., jet energy uncertainties. Jay had not considered this but agreed it sounded useful and worth exploring.

Adoption path and upstream tools (Giordon Stark, Jonas Rembser)

Giordon raised the concern that most analysers do not use pyhf or RooFit directly, but go through tools such as TRExFitter or cabinetry, and that without person-power on the upstream side, a good idea may not gain traction. Jay proposed a staged approach: keep the upstream tools unchanged initially, provide a converter that turns a standard HistFactory spec into a HistFactory V2 spec with sensible defaults (no extra anchor points, no interpolation-uncertainty NP), and only later expose the additional features. Jonas agreed that a "big rewrite" is to be avoided and that this iterative path is sensible, while noting that fully exploiting GP features (interpolation NP, MC-stat absorption) will eventually require non-trivial changes downstream.

XRooFit as an entry point (Will Buttinger)

Will proposed that XRooFit could serve as an early testing ground: a script could automatically replace PiecewiseInterpolation nodes in any HistFactory-built RooWorkspace with the new GP-based node, and XRooFit's interface could expose adding extra anchor points. TRExFitter, which already uses XRooFit, could then experiment with the functionality without waiting for a HistFactory schema change.

Will also asked about the timeline for a RooAbsReal-derived class (tentatively GPInterpolation, parallel to PiecewiseInterpolation) and offered to liaise on the API. Jonas confirmed he plans to start work on this after CHEP, likely in June/July, in two stages: first a Gaussian process function/class in ROOT::Math, then the RooFit operator built on top. The higher-level HistFactory API is a separate, later question, since the existing HistFactory XML schema is essentially frozen.

Serialisation and naming (Carsten Burgard)

Carsten suggested that the spec should not hard-code the distinction between histosys and GP_histosys. A more future-proof design would introduce a generic "multidimensional modifier" type with the GP variant as one subtype, leaving room for other multidimensional approaches (e.g. analytical parameterisations) to coexist with the same syntax and to factorise naturally with each other while allowing crosstalk within each block. Jay agreed and noted that additional metadata (kernel choice, prior choice, algorithm) can be attached per modifier.

Posterior uncertainty as a nuisance parameter (Jonas Rembser)

Jonas asked whether a nuisance parameter representing the GP posterior uncertainty would not be largely degenerate with the underlying nuisance parameters themselves, leading to fit instabilities. Jay confirmed this has not yet been studied and noted that the proposal in his slides (an extra γ-type degree of freedom modulated by the GP covariance, depending on α) is a suggestion from Lukas's original work that needs concrete investigation. Jonas observed that, combined with encoding MC-stat uncertainty as anchor-point noise, this could in principle absorb some of the existing MC-stat NPs into a smaller set of interpolation-uncertainty NPs — an interesting direction.

Parameterised anchor points and performance (Will Buttinger, Jonas Rembser)

Will asked whether the value of the function at an anchor point could itself be parameterised by another nuisance parameter (e.g. an MC-stat NP), and whether this would be too expensive given the GP evaluation. Jay had not tested this; Jonas was not too worried because GPs are generally fast to evaluate. Jay reported preliminary timing numbers showing GP evaluation roughly an order of magnitude faster than the comparable HistFactory computation in his small toy studies, so the combined cost (HistFactory prior + GP correction) should still be manageable. Benchmarking once a ROOT prototype exists was agreed as a useful next step.

Action Items and Next Steps

  • Jay: continue prototype work in pyhf; explore PCA-style dimensionality reduction; investigate using the GP posterior covariance as a nuisance parameter; study encoding MC-stat uncertainty as anchor-point noise.
  • Will and Jay: discuss offline an XRooFit-based prototype that swaps PiecewiseInterpolation for a GP-based node in existing RooWorkspaces; potentially trial it in Jay's analysis.
  • Jonas: implement a Gaussian process function/class in ROOT::Math and a corresponding RooAbsReal-derived class (provisionally GPInterpolation) after CHEP, targeting June/July; coordinate API with Will.
  • Generalise the V2 spec design so that GP_histosys is one subtype of a broader "multidimensional modifier" category, with kernel/prior/algorithm metadata per modifier.
  • Benchmark GP-based interpolation against HistFactory V1 once the ROOT prototype is available.

Next Meeting

Wednesday, 11 June 2026 (May is occupied by CHEP rehearsals and the CHEP conference itself; the two-week cadence resumes after that).

There are minutes attached to this event. Show them.
    • 15:00 → 15:05
      News and updates 5m
    • 15:05 → 15:30
      Updates on HistFactory v2 25m
      Speaker: Jay Ajitbhai Sandesara (University of Wisconsin (US))
    • 15:30 → 16:00
      Round table discussion 30m