Speakers
Description
Neural Simulation Based Inference is a fast-moving field with many ongoing efforts to share these promising methods with the wider HEP community. Whilst the specific NSBI methods that will eventually find adoption in actual analyses are not yet known, it is clear that most approaches face a set of common challenges. Foremost, the reliance on many large neural networks that train on large datasets to provide bias-free estimations leads to a considerable computational demand compared to traditional binned approaches.
Firstly, the NEEDLE project aims to provide a flexible framework that takes care of the common boilerplate shared by NSBI tools. This includes standardized model management with PyTorch and Lightning, flexible task-based orchestration with law and experiment tracking. In addition, ready-to-use data ingestion modules written with dask allow for out-of-memory computations for both root and parquet data formats. These modules can be used together in a finished workflow or used individually to accommodate existing code. The goal is to facilitate the deployment of NSBI methods without imposing strong constraints on the specific implementations.
Second, alongside the orchestration infrastructure, a companion benchmarking library to systematically evaluate generative models on toy datasets is being developed. Its goal is to help establish best-practices for the selection and configuration of density estimators. The investigation of powerful and expressive generative models is motivated by their increasing adoption for NSBI in fields such as cosmology and astro-particle physics.
In this contribution, we present an overview of how NEEDLE will provide a standard and flexible framework for NSBI deployment on HPC with minimal constraints, together with a toolbox of new generative models.