Fourth MODE Workshop on Differentiable Programming for Experiment Design

23 Sept 2024, 09:00 → 25 Sept 2024, 18:10 Europe/Madrid

Valencia (Spain)

Pietro Vischia (Universidad de Oviedo and Instituto de Ciencias y Tecnologías Espaciales de Asturias (ICTEA)), Tommaso Dorigo (Universita e INFN, Padova (IT)), Pablo Martinez Ruiz Del Arbol (Universidad de Cantabria and CSIC (ES)), Roberto Ruiz De Austri

MODE (for Machine-learning Optimized Design of Experiments) is a collaboration of physicists and computer scientists who target the use of differentiable programming in design optimization of detectors for particle physics applications, extending from fundamental research at accelerators, in space, and in nuclear physics and neutrino facilities, to industrial applications employing the technology of radiation detection.

Our aim to develop modular, customizable, and scalable, fully differentiable pipelines for the end-to-end optimization of articulated objective functions that model in full the true goals of experimental particle physics endeavours, to ensure optimal detector performance, analysis potential, and cost-effectiveness.

This workshop is part of the activities of the MODE Collaboration and of the SIVERT project.

Location:

The 2024 workshop will take place in Valencia following the third workshop in 2023 at Princeton University in Princeton, NJ, USA, the 2nd workshop in Crete in 2022 and 1st workshop in Louvain-La-Neuve in 2021.

 

Keynote Speakers:

• Danilo Rezende (DeepMind), webpage
• Andrea Walther (Humboldt Universität zu Berlin) webpage
• Riccardo Zecchina (Università Bocconi) webpage

 

Organizing Committee:

• Tommaso Dorigo (INFN-Padova)
• Christian Glaser (Uppsala Universiteit)
• Pablo Martinez Ruiz del Arbol (Universidad de Cantabria)
• Roberto Ruiz de Austri (IFIC, CSIC-UV)
• Pietro Vischia (Universidad de Oviedo and ICTEA)

 

Scientific Advisory Committee:

• Atilim Gunes Baydin (University of Oxford)
• Kyle Cranmer (University of Wisconsin)
• Julien Donini (Université Clermont Auvergne)
• Piero Giubilato (Università di Padova)
• Gian Michele Innocenti (CERN)
• Michael Kagan (SLAC)
• Riccardo Rando (Università di Padova)
• Kazuhiro Terao (SLAC)
• Andrey Ustyuzhanin (SIT, HSE Univ., NUS)
• Christoph Weniger (University of Amsterdam)

 

Local Organizing Committee

• Cesar Domingo (IFIC, CSIC-UV) 
• Gabriela Llosá (IFIC, CSIC-UV) 
• Roberto Ruiz de Austri (IFIC, CSIC-UV)
• José Salt (IFIC, CSIC-UV)
• Michel Sorel (IFIC, CSIC-UV) 
• Emma Torró (IFIC, CSIC-UV)
• Miguel Villaplana (IFIC, CSIC-UV)

 

Funding agencies:

This workshop is partially supported by:

• the joint ECFA-NuPECC-APPEC Activities (JENAA);
• the National Science Foundation cooperative agreement PHY-2323298 (IRIS-HEP);
• the Ministerio de Ciencia y Innovación, under the network RED2022-134769-T (COMCHA);
• the projects PROMETEO/2022/69 and PROMETEO/2022/70 from the Generalitat Valenciana;
• the Ministerio de Ciencia y Innovación and by the European Union NextGenerationEU/PRTR under the grant RYC2021-033305-I (NeuroModelling);
• the Particles journal.

 

 

 

 

Organized by:

 

Monday 23 September

09:00 → 09:30 Registration

09:30 → 10:00 Introduction

Convener: Dr Pietro Vischia (Universidad de Oviedo and Instituto de Ciencias y Tecnologías Espaciales de Asturias (ICTEA))

2024-09-25_MODEWorkshopClosing_vischia.pdf

HTML slides (preferred, animations work here)

09:30 Welcome

Speaker: Dr Pietro Vischia (Universidad de Oviedo and Instituto de Ciencias y Tecnologías Espaciales de Asturias (ICTEA))

09:31 Welcome from Local Organisers

Speakers: Jose Salt (Instituto de Fisica Corpuscular (IFIC) - Universidad de Valencia), Roberto Ruiz De Austri

Welcome_MODE_21Sept2024.pdf

09:35 Introduction and scope of the Workshop

Speaker: Tommaso Dorigo (INFN Padova and LTU)

4th MODE Workshop.pdf

10:00 → 11:00 Keynotes

Convener: Dr Pietro Vischia (Universidad de Oviedo and Instituto de Ciencias y Tecnologías Espaciales de Asturias (ICTEA))

10:00 Learning Without Labels

This presentation focuses on unsupervised representation learning. We first introduce the concept of representation learning, contrasting it with supervised learning. We then discuss several approaches to unsupervised representation learning, including those based on autoencoders, discriminators, contrastive and generative methods. Next, we shift our focus to generative models, discussing their foundations and different types of generative models. We highlight the relationship between unsupervised learning and the statistical, causal, and physical levels of modeling. Finally, we discuss the utility of unsupervised learning in a variety of downstream tasks, including image recognition, object detection, and reinforcement learning.

Speaker: Dr Danilo Rezende (Google Deep Mind)

10:50 Question time

11:00 → 11:30 Coffee Break

11:30 → 12:50 Particle Physics

Conveners: Emma Torro Pastor (Univ. of Valencia and CSIC (ES)), Dr Pietro Vischia (Universidad de Oviedo and Instituto de Ciencias y Tecnologías Espaciales de Asturias (ICTEA))

11:30 Cheetah: Bridging the Gap Between Machine Learning and Particle Accelerator Physics with High-Speed, Differentiable Simulations

Machine learning has emerged as a powerful solution to the modern challenges in accelerator physics. However, the limited availability of beam time, the computational cost of simulations, and the high-dimensionality of optimisation problems pose significant challenges in generating the required data for training state-of-the-art machine learning models. In this work, we introduce Cheetah, a PyTorch-based high-speed differentiable linear beam dynamics code. Cheetah enables the fast collection of large data sets by reducing computation times by multiple orders of magnitude and facilitates efficient gradient-based optimisation for accelerator tuning and system identification. This positions Cheetah as a user-friendly, readily extensible tool that integrates seamlessly with widely adopted machine learning tools. We showcase the utility of Cheetah through five examples, including reinforcement learning training, gradient-based beamline tuning, gradient-based system identification, physics-informed Bayesian optimisation priors, and modular neural network surrogate modelling of space charge effects. The use of such a high-speed differentiable simulation code will simplify the development of machine learning-based methods for particle accelerators and fast-track their integration into everyday operations of accelerator facilities.

Speaker: Jan Kaiser

20240923 Cheetah MODE Workshop.pdf

11:50 Identification of Particle Tracks in CMS with Neuromorphic Computing

Using a spiking neural network and a modeling of the silicon tracker for the CMS upgraded detector, we demonstrate the unsupervised learning application of identification of charged particle tracks in presence of background, and characterize the detection efficiency, fake rate, and differentiation of output signals for particles of different momenta and charge.

Speaker: Dr Emanuele Coradin (University of Padova)

MODE_valencia_Coradin.pdf

MODE_valencia_Coradin.pdf

MODE_valencia_Coradin.pptx

12:10 Machine Learning approach to Beam Induced Background Shield at 3 TeV Muon Collider

Muon collisions are considered a promising mean for exploring the energy frontier, leading to a detailed study of the possible feasibility issues. Beam intensities of the order of $10^{12}$ muons per bunch are needed to achieve the necessary luminosity, generating a high flux of secondary and tertiary particles that reach both the machine elements and the detector region. To limit the impact of this background on the physics performance tungsten shieldings are being studying. A machine learning-based approach to the geometry optimization of these shieldings will be discussed.

Speaker: Luca Castelli

mode_meeting.pdf

12:30 Stable neural network models for calorimeter optimization

The rapid development of ML and AI applications requires training a large number of models. One of the ways to organize training of them is the automated machine learning (AutoML) approach, where there is no human control over the training result. A crucial prerequisite for AutoML is the stability of the training model incorporated within it. This study presents an approach to identifying the training model of artificial neural networks, which achieves a specified convergence of the solution. The impact of sampling the training data and the initialization of the model weights are considered. The proposed model selection method extends conventional hyperparameter search approaches by identifying a stable model that meets a specified stability criterion. The results of this approach are shown for the optimization of an electromagnetic calorimeter.

Speaker: Alexey Boldyrev (NRU Higher School of Economics (Moscow, Russia))

Stable NN models for calorimeter optimization MODE_Workshop_23_09_2024.pdf

13:00 → 14:30 Lunch break

14:30 → 15:30 Keynotes

Convener: Dr Pietro Vischia (Universidad de Oviedo and Instituto de Ciencias y Tecnologías Espaciales de Asturias (ICTEA))

14:30 Collision Resistance in random Neural Network-Based Hash Functions

Contemporary post-quantum cryptographic protocols rely on worst-case intractability assumptions and consist of multiple intricate steps. In contrast, in this talk we shall explore a model system that directly addresses fundamental computational challenges and that can be mapped on a random neural networks.

We investigate the collision resistance property of a specific class of neural networks, positing that it is difficult to find two distinct sets of weights that produce the same labels for a random data set. Our analysis demonstrates this by upper bounding the local entropy as a function of the distance between collision pairs, revealing the emergence of an overlap gap property—a phenomenon widely considered a significant obstacle for efficient algorithms, including quantum annealing. These theoretical results are corroborated by numerical experiments employing approximate message passing algorithms and simulated annealing, both of which fail well before the predicted thresholds. Our findings suggest potential applications in contemporary cryptography, where these neural networks can be utilized to develop secure cryptographic hash functions.

Speaker: Prof. Riccardo Zecchina (Università Bocconi)

valencia_2024.pdf

15:20 Question time

15:30 → 16:10 Computer Science

Convener: Lisa Kusch (TU Eindhoven)

15:30 Algorithmic Derivatives of Electromagnetic Shower Simulations

Applying algorithmic differentiation to particle simulations like Geant4 would allow us to evaluate derivatives of simulation outputs with respect to inputs, e.g. of the mean energy depositions in calorimeter layers with respect to geometry parameters. Such derivatives could become instrumental for a number of application like detector optimization or parameter fitting in HEP. However, besides the technical challenge of applying AD to over one million lines of code, there is a number of mathematical/statistical challenges: Does the high density of discontinuities, induced for example by 'if' and 'while' statements in the code, cause problems in the derivative computation? Are we allowed to treat random numbers like constants with respect to AD? How large is the error when the mean pathwise derivative is evaluated as a proxy for the actual derivative of expectancies computed by Monte-Carlo algorithms?

To approach these questions, we have applied the operator-overloading AD tool CoDiPack to the compact G4HepEm/HepEmShow package for the simulation of electromagnetic showers in a simple sampling calorimeter. After disabling multiple scattering in the simulation, our pathwise gradient estimator approximates derivatives of energy depositions with only a small bias, which proved unproblematic in a simple optimization study. In this talk, we will report on our methodology and give an overview on our findings.

Speakers: Max Aehle (Fachhochschule Worms (DE)), Max Aehle

max-aehle-ad-for-g4hepem.pdf

15:50 Exploring End-to-end Differentiable Charged Particle Tracking

Current state-of-the-art in charged particle tracking follows a two-step paradigm where a graph neural network optimizes an intermediate prediction-loss during training and is later combined with a discrete, non-differentiable, optimization step during inference, constructing disconnected track candidates. In this talk, we introduce and assess a novel end-to-end differentiable tracking strategy. We use edge-classifying graph neural networks with differentiable combinatorial components, enabling direct optimization of the task-loss for discrete assignments. We provide further insights into the optimization processes and learned solutions, demonstrating similarities and limitations of two-step and end-to-end optimization. Finally, we demonstrate through a proof of concept that our approach can encode additional constraints or objective functions of downstream tasks, enabling the optimization of tracking solutions that meet specific performance criteria.

Speaker: Tobias Kortus (University of Kaiserslautern-Landau (RPTU))

tobias-kortus-e2e-tracking.pdf

16:10 → 16:35 Coffee Break

16:35 → 18:15 Astroparticle

Conveners: Christian Glaser (Uppsala University), Roberto Ruiz De Austri

16:35 NuRadioOpt: Optimization of Radio Detectors of Ultra-High Energy Neutrinos through Deep Learning and Differential Programming

Detection of neutrinos at ultra-high energies (UHE, E >$10^{17}$eV) would open a new window to the most violent phenomena in our universe. However, owing to the expected small flux of UHE neutrinos, the detection rate will be small, with just a handful of events per year, even for large future facilities like the IceCube-Gen2 neutrino observatory at the South Pole.
In this contribution, we will discuss how to substantially enhance the science capabilities of UHE neutrino detectors by increasing the detection rate of neutrinos and improving the quality of each detected event, using recent advances in deep learning and differential programming. First, we will present neural networks replacing the threshold-based trigger foreseen for future detectors that increase the detection rate of UHE neutrinos by up to a factor of two. Second, we will outline and present preliminary results towards an end-to-end optimization of the detector layout using differential programming and deep learning, which will improve the neutrino direction and energy determination. We will present a new Likelihood Ansatz that allows estimating the reconstruction resolution by calculating the Fisher information which removes the need for developing and tuning reconstruction algorithms for varying detector layouts.
We estimate that the expected improvements will be equivalent to building an up to three times larger detector, accelerating the research field by a decade.

Speaker: Christian Glaser (Uppsala University)

NuRadioOpt_Glaser.pdf

NuRadioOpt_Glaser.pptx

16:55 Machine learning based design optimization for the search of neutrinoless double-beta decay with LEGEND

Cosmic muon interactions leading to the in-situ production of long-lived radioisotopes may introduce a significant background in the context of rare event searches conducted deep underground. Specifically, the delayed decay of $^{77(m)}$Ge emerges as the primary contributor from in-situ cosmogenic sources for the neutrinoless double-beta decay search with $^{76}$Ge. The future LEGEND-1000 experiment, aiming for a ton-scale setup, necessitates a stringent requirement of a total background less than $10^{-5} \rm cts/(keV\cdot kg \cdot yr)$. Neutron backgrounds are closely tied to factors such as laboratory depth, shielding material, and cryostat design. The incorporation of passive neutron moderators results in a reduced background contribution. In order to determine the most effective shield design, computationally intensive Geant4 Monte Carlo simulations need to be generated multiple times to probe the high-dimensional parameter spaces. Traditional Monte Carlo simulations, however, may prove time-consuming and challenging when addressing full optimization across numerous parameter spaces. This renders conventional methods, such as grid searches, computationally infeasible. Machine learning emerges as a valuable tool, not only for accelerating common modeling but also for minimizing the reliance on computationally expensive standard Monte Carlo methods. We outline a study of a Multi-Fidelity Gaussian Process combined with a Conditional Neural Process, showcasing its application in a small-scale context based on various neutron moderator configurations. The approach presented holds the potential for adaptability in exploring alternative detector shielding designs for $^{76}$Ge experiments, such as LEGEND.

Speaker: Dr Ann-Kathrin Schuetz (Lawrence Berkeley National Laboratory)

LEGEND_ML-MODE-Sept24_ASchuetz.pdf

17:15 Status of GENETIS

GENETIS aims to use AI to find optimal designs of instruments for greater science outcomes. Initially, we are using genetic algorithms to evolve optimal antenna designs for the detection of astrophysical neutrinos and is building a prototype of what is the first antenna evolved for a science outcome. The Nebulous spin-off project is building antenna designs from building blocks “LEGO”-style rather than evolving parameters of a preconceived design. Since the previous MODE workshop, GENETIS and Nebulous have grown and strengthened in the areas of AI, engineering, computer science, astrophysics, and connections to industry. I will review GENETIS and Nebulous results to date, outcomes of a Blue Sky Studies workshop to be held at JPL in August, and plans for the future.

Speaker: Amy Connolly (The Ohio State University)

2024_Mode.pdf

17:35 Simultaneous high-dimensional calibration with differentiable simulations for a liquid argon time projection chamber

The fidelity of detector simulation is crucial for precision experiments, such as DUNE which uses liquid argon time projection chambers (LArTPCs). We can improve the detector simulation by performing dedicated calibration measurements. Using conventional calibration approaches, typically we are only able to tackle individual detector processes per measurement. However, the detector effects are often entangled in the measured detector output, particularly in LArTPCs. In this talk, we will present differentiable simulation solutions with gradient-based optimization for improving the modeling of both the charge and light signals from LArTPCs. The use of the differentiable simulator allows in-situ calibration which provides natural consistency between the calibration measurements and simulation application. This work also paves a way towards developing an "inverse detector simulation" for mapping the detector output to detector physics quantities of interest.

Speaker: Yifan Chen (SLAC National Accelerator Laboratory (US))

Mode_Valencia_YifanChen_2024_09_23_diffsim.pdf

17:55 TAMBO: Searching for Tau Neutrinos in the Peruvian Andes

The detection of high-energy astrophysical neutrinos by IceCube has opened a new window on our Universe. While IceCube has measured the flux of these neutrinos at energies up to several PeV, much remains to be discovered regarding their origin and nature. TAMBO is a next-generation neutrino observatory specifically designed to detect tau neutrinos in the 1-100 PeV energy range, enabling tests of neutrino physics at high energies and the characterization of astrophysical neutrino sources. The observatory will comprise an array of water Cherenkov and plastic scintillator detectors deployed on the face of the Colca canyon in the Peruvian Andes. This unique geometry will facilitate a high-purity measurement of astrophysical tau neutrino properties. In this talk, I will present the prospects of TAMBO in the context of next-generation neutrino observatories and provide an overview of its current status.

Speaker: Dr Alfonso Andres Garcia Soto (IFIC)

tambo.pdf

18:15 → 19:30 Free time

19:30 → 20:30 Social events: Concert

Dinner Venue

Google maps link

Tuesday 24 September

09:00 → 10:00 Muography

Convener: Pablo Martinez Ruiz Del Arbol (Universidad de Cantabria and CSIC (ES))

09:00 Optimisation of muon tomography scanners for border controls using TomOpt

TomOpt is a software package designed to optimize the geometric configuration and specifications of detectors intended for tomography using cosmic-ray muon scattering. Differentiable programming is utilized by the software to model muon interactions with detectors and scanned volumes, infer volume properties, and perform loss minimization in an optimization cycle. We introduce the implementation of a case study related to cargo scanning at border controls.

Speaker: Zahraa Zaher

Zahraa_mode24.pdf

09:20 Generative Adversarial Neural Networks for muography simulation: image prediction

Generative Adversarial Neural Networks (GANN) are used to simulate the multiple scattering of muons crossing matter. In previous works, a GANN was designed and trained, successfully predicting the angular and spatial deviation distributions including their correlations. In this work we show that GANNs can be so good at this task that correct POCA images can be reconstructed from their randomly generated samples.

Speaker: Pablo Martinez Ruiz Del Arbol (Universidad de Cantabria and CSIC (ES))

PresentationModeWorkshop.pdf

PresentationModeWorkshop.pptx

09:40 Segmentation and anomaly detection on Muon Tomography applied to the inspection of containers

Muon Cargo is a project funded by the Spanish Port Authority aiming at installing a Muography portal for container inspection in the port of Santander. This talk offers a panoramic of the status of the project focusing on the development of two AI algorithms: a YOLOv8 based system to perform semantic segmentation on POCA-based images, and a Variational Autoencoder to identify unsual, unexpected geometrical patterns in the containers.

Speaker: Carlos Diez (Muon Tomography Systems S.L.)

20240924_MODE_MuonSystems.pptx

10:00 → 11:00 Keynotes

Convener: Dr Pietro Vischia (Universidad de Oviedo and Instituto de Ciencias y Tecnologías Espaciales de Asturias (ICTEA))

10:00 Derivative-based optimization for applications in physics

The provision of exact and consistent derivative information is important for numerous applications arising from optimization purposes as for example optimal control problems. However, even the pure simulation of complex systems may require the computation of derivative information. Implicit integration methods are prominent examples for this case.

The talk will present the technique of algorithmic (or automatic) differentiation (AD) to compute exact derivative information for function evaluations given as computer programs. This includes a short overview of the history of AD and a description of the main variants of AD, namely the forward mode to compute sensitivities as well as the reverse mode for the provision of adjoints. A discussion of complexity estimates follows yielding the important cheap gradient result. Subsequently, I will present two applications of derivative-based optimization in the field of physics. First, the parameter identification of material parameters for an ultrasound application is discussed. This includes also the question of an optimal experimental design. Second, we will examine resonance phenomena in the field of photonics. Here, capturing sensitivities is crucial for the investigation and optimization of exceptional-point-based applications,
such as optical sensors.

Speaker: Prof. Andrea Walther (Humboldt Universität Berlin)

talk_walther-2.pdf

talk_walther.pdf

10:50 Question time

11:00 → 11:30 Coffee Break and Group Photo

11:30 → 12:50 Particle Physics

Convener: Dr Pietro Vischia (Universidad de Oviedo and Instituto de Ciencias y Tecnologías Espaciales de Asturias (ICTEA))

11:30 Magnet Design Optimization with Supervised Deep Neural Networks

...

Speaker: Florian Wolfgang Stummer (University of London (GB))

2024-09-24_StummerFlorian_ModeWorkshop2024.pdf

11:50 A Surrogate Model to Optimize Injection Efficiency in PSI muEDM Experiment

...

Speaker: Ritwika Chakraborty (PSI)

MODEWorkshop2024.pdf

12:10 Towards the optimization of a Muon Collider Calorimeter

Setup design plays a pivotal role in experiment development, particularly in high-energy physics, where vast temporal and spatial scales dictate the course of research for decades. Our research, embedded in the MODE Collaboration, aims to generalize Machine Learning tools for creating a differentiable pipeline capable of suggesting optimal configurations for the Muon Collider Electromagnetic Calorimeter geometry. In our presentation we outline the structure of our pipeline, emphasizing the methods employed to ensure full code differentiability. Our primary focus lies in maximizing the reconstruction efficiency of photons amidst Beam-Induced background from muon decays. The approach relies on three core blocks: (I) Signal Event Generator: Responsible for generating signal events; (II) Background Generator: Focused on simulating background events; (III) Reconstruction Algorithm: Adapting the DeepJetCore Object Condensation framework. The talk includes a showcase of performance tests for each core block, shedding light on their efficacy. Additionally, we provide insights into the current status and challenges encountered in implementing the complete pipeline.

Speaker: Federico Nardi (Universita e INFN, Padova (IT) - LPC Clermont)

MODE_SEPT_2024.pdf

12:30 Studies on detector optimisation through end-to-end surrogate models including discrete parameters.

Speaker: Jan Kieseler (KIT - Karlsruhe Institute of Technology (DE))

20240924_Valencia.pdf

13:00 → 14:30 Lunch break

14:30 → 15:50 Astroparticle

Conveners: Christian Glaser (Uppsala University), Roberto Ruiz De Austri

14:30 Application of machine learning techniques to search dark matter with ANAIS-112

Understanding the nature of dark matter is one of the greatest challenges faced by Particle Physics in the XXI century. To date, the only hint about a positive identification of the dark matter comes from the DAMA/LIBRA experiment in the Gran Sasso National Laboratory (Italy). For more than 20 years, it has observed an annual modulation in the low-energy detection rate of its NaI(Tl) crystals, compatible with that expected for dark matter particles from the Milky Way’s halo. This signal is in strong tension with the negative results of other very sensitive experiments, but since they use different target materials, the comparison depends on the models assumed for the dark matter particle and its velocity distribution in the galactic halo. The ANAIS-112 (Annual modulation with NaI(Tl) scintillators) experiment, operating 112.5 kg of NaI(Tl) at the Canfranc Underground Laboratory (Spain) since 3 August 2017, intends to provide a model independent test of the signal reported by DAMA/LIBRA, using the same target and technique. Due to the low probability of dark matter interacting with the detector, ultra-low radioactive background conditions are imperative. However, even so, the region of interest (1-6 keV) is dominated by noise events not originating in the scintillation crystal, but mainly in the photomultiplier tubes coupled to the crystals. In order to discriminate these noise events from bulk scintillation events, developing robust filtering protocols is a must. Profiting from the application of machine learning techniques in ANAIS-112, the efficiency of selecting good events between 1 and 2 keV has increased by 30% compared to the use of standard cuts, and the background level has been reduced by almost 20%, resulting in an increase in sensitivity to the annual modulation signal. Last ANAIS-112 results using machine learning filtering procedures will be presented, as well as the experimental status and prospects. These results lead international efforts in testing the DAMA/LIBRA signal, and are compatible with the absence of modulation and incompatible with DAMA/LIBRA for a sensitivity of almost 3σ C.L. (using 3-year exposure) and above 4σ (6-year exposure), with the potential to reach 5σ level by the end of 2025.

Speaker: Iván Coarasa Casas (CAPA, University of Zaragoza, Spain)

FourthMODEWorkshop2024_ANAIS112_IvanCoarasa_vdef.pdf

14:50 Optimization of the DSA 2000 radio interferometer

In this presentation, I will discuss the forward modeling of the DSA 2000 radio interferometer, an array set to exceed the capabilities of any existing or planned radio interferometer. Our approach leverages forward modeling to design and validate the system, ensuring it meets scientific requirements, budget constraints, and computational feasibility. I will introduce our JAX-based forward modeling framework, highlighting its distributive, auto-differentiation, and high-level programming capabilities, which facilitate rapid prototyping and development. This talk will also showcase several novel results from our forward modeling efforts. This marks the first instance where forward modeling has been applied with such detail to characterize a radio interferometer both before and after construction.

Speaker: Joshua Albert (Caltech)

Optimising the DSA-2000-1.pdf

15:10 Applications of Information Field Theory in Astroparticle Physics

Information Field Theory (IFT) offers a powerful framework for the analysis of experimental data. The fundamental objective of IFT is the reconstruction of continuous fields from noisy and sparse data. By combining Bayesian probabilities with computational techniques from quantum field theory and statistical mechanics, IFT allows for efficient inference in high-dimensional problems.
In this talk, we discuss the application of IFT in the context of astroparticle physics. First, we present its use for the calibration of the newly installed radio detector upgrade of the Pierre Auger Observatory, the world's largest cosmic-ray observatory. By analyzing data from drone flights, the radio response patterns are calibrated using IFT. Second, we present a novel application of IFT to infer properties of the Galactic magnetic field using the arrival directions of ultra-high-energy cosmic rays in a simplified toy model.

Speaker: Frederik Krieger (RWTH Aachen University)

mode_frederik_krieger.pdf

15:30 A surrogate model for the generation of radio pulses from neutrinos for IceCube-Gen2

The planned IceCube-Gen2 radio neutrino detector at the South Pole will enhance the detection of cosmic ultra-high-energy neutrinos. It is crucial to make use of the time available until its construction to optimize the detector design. A fully differentiable pipeline, from signal generation to detector response, would allow for the application of gradient descent techniques to explore the parameter space of the detector. In our work, we focus on the aspect of signal generation, and propose a deep learning architecture involving a diffusion model to generate radio signals from in-ice neutrino interactions conditioned on the shower energy and viewing angle. We evaluate the performance of our model in terms of the distributions of different signal characteristics, and we investigate whether the correct physical dependence on the viewing angle is learned.

Speaker: Philipp Pilar (Uppsala University)

RadioPulses_Pilar.pdf

16:00 → 16:30 Coffee Break

16:30 → 18:10 Nuclear Physics

Convener: Cesar Domingo Pardo (Instituto de Fisica Corpuscular (ES))

16:30 Automated Design of Quantum Experiments

...

Speaker: Mario Krenn (MPI)

Krenn_Valencia_v2.pdf

16:50 Automatic optimization of a Parallel-Plate Avalanche Counter with Optical Readout

We propose an optimization system for a Parallel-Plate Avalanche Counter with Optical Readout designed for heavy-ion tracking and imaging. Exploiting differentiable programming, we model the reconstruction of the position for different detector configurations and build an optimization cycle that minimizes an objective function. We analyze the performance improvement using this method, exploring the potential of these techniques with the ultimate goal of fully designing a tomography system based on neutrons.

Speaker: María Pereira

mode_workshop_2024_maria_pereira.pdf

mode_workshop_2024_maria_pereira.pptx

17:10 Utilizing Neuronal Networks to Enhance the Application of Metallic-Magnetic Calorimeters for X-ray Spectroscopy

Metallic-magnetic calorimeters (MMCs) - like the maXs-detector series developed within the SPARC collaboration - have become a promising new tool for high precision X-ray spectroscopy. Because of their unique working principles, MMCs combine several advantages over conventional energy- and wavelength-dispersive photon detectors. They can reach spectral resolving powers of up to $E / \Delta E \approx 6000$ (at $60\,\text{keV}$) [1] and at the same time, cover a broad spectral range of typically $1-100\,\text{keV}$. Combined with their excellent linearity [2] and a sufficiently fast rise time - e.g., for coincidence measurement schemes [3] - they are particularly well suited for fundamental physics research in atomic physics.
However, because of their high sensitivity, external sources of noise like physical vibrations lead to substantial measurement artifacts [5] and a shift from traditional analog to a digital signal processing was necessary to mitigate these effects. During several successful benchmark experiments [3, 4, 5, 6] a comprehensive signal analysis software framework was developed. Though, the setup and operation of the detectors, as well as the analysis of their signals using our software, requires a per pixel optimization of a multitude of parameters. Several steps cannot be automated by conventional means and require manual tuning. As future development of MMCs entails the addition of more and more pixels, this approach is no longer applicable.
Research in the field of artificial intelligence offers a plethora of novel solutions that could help simplify and improve the measurement scheme of MMCs on both the hardware and software level. In this work we present first results from the application of neuronal networks (NNs) for raw MMC pulse processing. Based on first experiments, we expect NNs to at least be on par with feature extraction capabilities of the currently used finite response filters, without extensive manual fine-tuning and in a wider range of applicability.

References
[1] J. Geist, Ph.D. Thesis, Ruprecht-Karls-Universität Heidelberg, Germany (2020)
[2] C. Pies et al., J. Low Temp. Phys. 167 (2012) 269–279
[3] P. Pfäfflein et al., Physica Scripta 97 (2022) 0114005
[4] M.O. Herdrich et al., X-Ray Spectrometry 49 (2020) 184–187
[5] M.O. Herdrich et al., Atoms 11 (2023) 13
[6] M.O. Herdrich et al., Eur. Phys. J. D 77 (2023) 125

Speaker: Mr Marc Oliver Herdrich (Helmholtz-Institute Jena, Germany)

MODE24_Herdrich.pdf

MODE24_Herdrich.pptx

17:30 Differentiable Deep Learning Surrogate Models applied to the optimization of the IFMIF-DONES facility

One of the primary challenges for future nuclear fusion power plants is understanding how neutron irradiation affects reactor materials. To tackle this issue, the IFMIF-DONES project aims to build a facility capable of generating a neutron source in order to irradiate different material samples. This will be achieved by colliding a deuteron beam with a lithium jet. In this work, within the DONES-FLUX project, Deep Learning Surrogate Models are applied to the design and optimization of the IFMIF-DONES linear accelerator. Specifically, Neural Operators are employed to predict deuteron beam envelopes along the longitudinal axis of the accelerator and neutron irradiation effects at the end, after the beam collision. This approach has resulted in models that are able of approximating complex simulations with high accuracy (mean absolute percentage errors of 1-5%) and significantly reduced inference time (ranging from 2 to 5 orders of magnitude) while being differentiable. The substantial speed-up factors enable the application of online Reinforcement Learning algorithms, and the differentiable nature of the models allows for seamless integration with differentiable programming techniques, facilitating the solving of inverse problems to find the optimal parameters for a given objective. Overall, these results demonstrate the synergy between Deep Learning models and differentiable programming, offering a promising collaboration among physicists and computer scientists to further improve the design and optimization of IFMIF-DONES and other accelerator facilities. This research will lay the foundations for future projects such as DONES+MAGIA, where optimization efforts with differentiable programming will be performed.

Speaker: Galo Gallardo Romero (Artificial Intelligence Engineer at HI Iberia)

MODE_2024.pptx

17:50 Optimal design for experimental production of exotic hypernuclei

In the HypHI project, which started in 2006 at GSI-FAIR, we aim to study proton- and neutron-rich hypernuclei produced in the ion-induced collisions. The successful observation of light hypernuclei in the 6Li – 12C collisions during our first experimental campaign in 2009 – 2010 has paved a new way to study these bound states of protons, neutrons, and hyperons [1]. For future experiments, both the Fragment Separator (FRS) of GSI and the Superconducting Fragment Separator (SuperFRS) of FAIR are needed to produce those exotic hypernuclei. A new experimental campaign was designed using solenoid magnet and the central detection system of WASA in the middle focal plane of FRS, which combines the hadron spectrometer capabilities of WASA with the high-resolution forward spectrometer capabilities of the second half of FRS. The experimental goal is to demonstrate that the production and study of hypernuclei can be achieved by this new experimental method [2]. In future experiments involving fragment separators and solenoid spectrometers to study proton- and neutron-rich hypernuclei, a thorough study of the optimal conditions of the experimental setup is crucial.

Our contribution to this workshop will be the presentation of the optimal conditions we obtained through optimal design of experiment already published in [3]. We will introduce the concept of optimal design, which aims to identify the most suitable experimental settings for the production of exotic hypernuclei. To achieve this, we evaluated different experimental variables and conditions needed to be set at SuperFRS, including: type of primary beam, type of secondary beam, energy beam, transmission efficiencies of the primary and secondary beams. Those conditions are then coupled to a QMD theoretical model of nuclear collisions for the hypernuclear production. Depending of the secondary beam species, beam energy and intensities, and the target materials, the production of proton- or neutron-rich hypernuclei are widely impacted. Through a multivariate analysis, the optimal experimental conditions for 40 different exotic hypernuclei were summarized.

In our presentation, we will delve into the details of our methodology, providing an explainable overview of how we arrived at these optimal conditions.

[1] C. Rappold et al., Hypernuclear spectroscopy of products from 6Li projectiles on a carbon target at 2AGeV, Nuclear Physics A. 913, p.170 (2013)
[2] T. Saito et al., New directions in hypernuclear physics, Nature Review Physics: 3, p. 803. (2021)
[3] C. Rappold and J. López-Fidalgo, Examination of experimental conditions for the production of proton-rich and neutron-rich hypernuclei, Physical Review C. 94, pp.044616 (2016)

Speaker: Samuel Escrig López (Instituto de Estructura de la Materia (IEM - CSIC))

Optimal design production hypernuclei - MODE2024.pdf

18:20 → 20:00 Poster session

18:25 Algorithmic differentiation in a granular calorimeter

Algorithmic differentiation (AD) allows to compute derivative of
computer-implemented function. Among other applications, such
derivatives are useful across domains for gradient-based design
optimization and parameter fitting. In the context of high-energy
physics, AD may allow to systematically improve detector designs based on end-to-end simulations of detectors. We have recently added
an important building block to this end by releasing a forward-mode
differentiated version of the Geant4 toolkit for the simulation of the
passage of particles through matter, validating derivatives for a
simple sampling calorimeter made up of square layers of absorber and
gap material.

In this poster, we use the differentiated version of Geant4 to simulate
a calorimeter made up of a 100x100x100 grid of sensitive voxels. This
allows to validate the derivatives investigated in our previous study,
namely, derivatives of energy depositions with respect to the primary
energy of incoming particles and geometric lengths of the detector
design, in this more complex setup.

Speaker: Xuan Tung Nguyen (INFN and RPTU)

MODE-poster.pdf

18:25 Computational Evaluation of Na3SbX3 (X =S, Se) for resistive switching memory devices for Neuromorphic Computing applications

Neuromorphic Computing draws inspiration from the brain. Resistive Random-Access Memory (ReRAM) is gaining attention for its potential use in neuromorphic computing, mimicking neural networks for efficient data processing. Recent advancements introduce innovative materials such as metal oxides (e.g., HfO₂, Ta₂O₅), perovskites, and 2D materials like graphene, which enhance the scalability, switching speed, and power efficiency of ReRAM devices. In our work, we have explored new chalcogens for ReRAM applications. These materials possess suitable optoelectronic attributes, enable multi-level states and greater synaptic plasticity, making them ideal candidates for future neuromorphic systems that require high-density, low-power, and brain-like information processing.

Speaker: Muhammad Awais

poster-MUHAMMAD AWAIS.pdf

18:25 Development and explainability of models for machine-learning-based signal reconstruction

Machine learning methods are being introduced to all stages of data reconstruction and analysis in various high energy physics experiments. We present the development and application of convolutional neural networks with modified autoencoder architecture. These networks are aimed at reconstructing the pulse arrival time and amplitude in individual scintillating crystals in the PADME experiment detectors. The network performance is discussed as well as the application of xAI methods for further investigation of the algorithm and improvement of the output accuracy.

Speaker: Kalina Dimitrova (University of Sofia - St. Kliment Ohridski (BG))

MODE_2024_Poster-1.pdf

18:25 Evo-SAN: Evolutionary Optimization and Co-design of Energy-efficient Neuromorphic Hardware Trigger for UHE Neutrino Detection

Speaker: Mr Alexander Kyuroson (Lulea Technical University)

MODE2024_Poster_A0_Final_v2.pdf

18:25 Improving BioDynamo's Performance using ROOT C++ Modules

Speaker: Mr Isaac Morales Santana (Universidad de Granada)

introducción 2.pdf

18:25 Machine Learning for Particle Identification in Real Time at the ATLAS Experiment

Machine learning algorithms have proven to be powerful tools for identifying and classifying different types of particles. This is especially useful in experiments like the ATLAS experiment at CERN. The large and complex amount of data generated from proton-proton collisions at the Large Hadron Collider (LHC) require advanced techniques to accurately identify various particle signatures for later distribution to different analyses.

In the current data-taking scenario, machine learning techniques are used to identify tau leptons and b quarks. A Recurrent Neural Network (RNN) has replaced the previous Boosted Decision Tree (BDT) for tau lepton identification in the ATLAS High-Level Trigger, significantly improving reconstruction efficiency. More advanced algorithms have also been developed for b-quark identification. The b-tagging algorithms implemented for Run 3 also improved to distinguish these jets from background jets created by single quarks and gluons.

Speaker: Marta Lanzac Berrocal (Univ. of Valencia and CSIC (ES))

Poster MODE Workshop-2.pdf

18:25 Machine learning in muography: Optimisation of muon detection systems for border control using TomOpt

Speaker: Mr Samuel Álvarez Lueje (Universidad de Oviedo)

Valencia MODE (841 x 1189 mm)-1.pdf

18:25 Multivariate two-sample tests from univariate integral probability measures

I will present and discuss several proposed metrics, based on integral probability measures, for the evaluation of generative models (and, more generally, for the comparison of different generators). Some of the metrics are particularly efficient to be computed in parallel and show good performances. I will first compare the metrics on toy multivariate/multimodal distributions, and then focus on HEP examples from the JetNet jet dataset. I will discuss the power of the tests and their implementation, taking the opportunity to discuss, more generally, the limitations in the usage of generative models in HEP.

Speaker: Samuele Grossi (Università degli studi di Genova & INFN sezione di Genova)

Poster_Valencia_MODE_2024.pdf

18:25 Neuromorphic Readout For Homogeneous Hadron Calorimeters

We investigate the transduction-less readout of light signals from hadronic showers in a homogeneous calorimeter by nanowires that can be arranged in a network, communicating through the time-encoding of light pulses, and offering fast, energy-efficient local computation and generation of informative high-level primitives for the precise measurement of shower energy and the identification of particle species using neuromorphic computing.

Speaker: Enrico Lupi (INFN Padova and University of Padova)

LupiEnrico_PosterMode_NCCal.pdf

18:25 Porting MADGRAPH to FPGA using High-Level Synthesis (HLS)

The escalating demand for data processing in particle physics research has spurred the exploration of novel technologies to enhance efficiency and speed of calculations. This study presents the development of a porting of MADGRAPH, a widely used tool in particle collision simulations, to FPGA using High-Level Synthesis (HLS).
Experimental evaluation is ongoing, but preliminary assessments suggest a promising enhancement in calculation speed compared to traditional CPU implementations. This potential improvement could enable the execution of more complex simulations within shorter time frames.
This study describes the complex process of adapting MADGRAPH to FPGA using HLS, focusing on optimizing algorithms for parallel processing. A key aspect of the FPGA implementation of the MADGRAPH software is reduction of the power consumption, which important implications for the scalability of computer centers and for the environment. These advancements could enable faster execution of complex simulations, highlighting FPGA's crucial role in advancing particle physics research and its environmental impact.

Speaker: Hector Gutierrez Arance (Univ. of Valencia and CSIC (ES))

Porting MADGRAPH to FPGA using High-Level Synthesis (HLS).pdf

18:25 Smart Web Services in the Web of things

In the field of the Web of Things (WoT), there has been significant progress in connecting diverse real-world objects, integrating them into the virtual realm, and ensuring their seamless interoperability. Achieving this objective necessitates a focus on developing intelligent web services capable of autonomously executing tasks, adapting to evolving object contexts, and user preferences. This article proposes a software framework for smart services that leverages the concepts of context and time, employing a reactive and proactive approach. By adhering to these principles, smart web services can react to present conditions and proactively anticipate unforeseen situations to make informed decisions.

Speaker: Nawel SEKKAL (University Abou Bekr Belkaid Tlemcen)

Anstract2.docx

Anstract2.pdf

18:25 Toward Particle ID in Granular Hadron Calorimeters

High granularity has become a desirable feature in hadron calorimeters after the parallel realizations that 1) the hadronic decay of boosted heavy particles could be successfully identified within fat jets, and 2) that particle flow techniques relying on detailed structure of the hadronic showers are an invaluable technique for event reconstruction. In this work we study if arbitrarily high granularity may also allow for a discrimination of the identity of hadrons of different kinds as generators of the observed showers. Using Geant4 we simulate a million-cell calorimeter module and seek discrimination of protons from charged pions and kaons. In our preliminary results we obtain significant discrimination power by analyzing detailed features of the topology of the energy deposition.

Speaker: Andrea De Vita (Universita e INFN, Padova (IT))

ModePoster_DeVita.pdf

18:25 Versal ACAP processing for ATLAS-TileCal signal reconstruction

Particle detectors at accelerators generate large amount of data, requiring analysis to derive insights. Collisions lead to signal pile up, where multiple particles produce signals in the same detector sensors, complicating individual signal identification. This contribution describes the implementation of a deep learning algorithm on a Versal ACAP device for improved processing via parallelization and concurrency. Connected to a host computer via PCIe, this system aims for enhanced speed and energy efficiency over CPUs and GPUs. In the contribution, we will describe in detail the data processing and the hardware, firmware and software components of the signal reconstruction of the Tile Calorimeter (TileCal) of the ATLAS detector which will be running in real time in the HL-LHC era. The contribution presents the implementation of the deep learning algorithm on Versal ACAP device, as well as the system for transferring data in an efficient way. In addition, the system integration tests and results from the tests with beam performed at CERN will be presented.

Speaker: Francisco Hervas Alvarez (Univ. of Valencia and CSIC (ES))

Versal ACAP processing for ATLAS-TileCal signal reconstruction.pdf

20:00 → 21:00 Free time

21:00 → 23:55 Social events: Social dinner

Dinner Venue

Google maps link

Wednesday 25 September

09:00 → 10:00 Muography

Convener: Pablo Martinez Ruiz Del Arbol (Universidad de Cantabria and CSIC (ES))

09:00 Muon momentum estimation in scattering tomography

Muon scattering tomography allows for the imaging of the density of unknown volumes through the measurement of the incoming and outgoing tracks scattering angle. One significant source of imprecision of the technique comes from the dependence of muon momentum on the multiple scattering process that muons undergo in the material. This can be alleviated by including dedicated momentum measurement devices to the experiment, with a potential cost and interference with the scattering measurement.

An alternative consists in leveraging the scattering information itself, through passive scattering material downstream of the measured volume. In this talk, we present a comprehensive study of diverse machine learning algorithms for this regression task. From a classical feature engineering with a fully connected network, to more advanced architectures such as recurrent and graph neural networks, and transformers. Several real-life requirements are considered, such as the inclusion of hit reconstruction efficiency and resolution, and the need for a momentum resolution prediction that can improve reconstruction methods like PoCA or MLEM.

Speaker: Mr Florian Bury (University of Bristol)

MODE 4th workshop.pdf

09:20 Improving Material Identification in Muon Scattering Tomography with Machine Learning.

Machine learning holds significant potential for improving Muon Scattering Tomography (MST) material identification. However, the complexity of acquiring sufficient MST data for machine learning applications remains a significant challenge. To circumvent this, there is a growing interest in creating MST synthetic datasets using Geant4, a software that can accurately simulate muon-material interactions while mirroring real-world scenarios. Yet, traditional time-intensive scene generation significantly limits the scalability of data generation pipelines. Our research introduces scalable synthetic data generation advancements with detailed 3D scenes using Geant4. Then, we propose a novel data workflow specific to MST to simulate scenes with different materials and shapes to evaluate the effectiveness of deep learning techniques in identifying material properties. Our experimental results yield enhanced material identification while highlighting the current challenges in designing deep learning algorithms for muon tomography applications.

Speaker: Angel Bueno Rodriguez (German Aerospace Center (DLR))

modeBueno.pdf

09:40 Muographic Image Up-Sampling with Machine Learning for Built Infrastructure Applications

In the civil engineering industry, there is an increasing demand for innovative non-destructive evaluation methods, especially for critical infrastructure such as bridges, as current techniques fall short. Muography, a non-invasive technique, constructs three-dimensional density maps by detecting the interactions of naturally occurring cosmic-ray muons within the scanned volume. Due to their high momenta, cosmic-ray muons can penetrate to depths where other techniques falter, and since the source is natural, there are no safety risks involved. However, the reliance on a natural source imposes a constraint on the muon flux, resulting in prolonged acquisition times and noisy image reconstructions. Framing the problem as an image processing task, we employ a convolutional-based supervised machine learning approach to map 2-dimensional under-sampled muography images to their high-sampled counterpart. We propose using a conditional generative adversarial network (cGAN), trained on data produced by Geant4 Monte-Carlo simulations, to learn the generalized mappings and investigate up-sampling limits. A discussion on the design and operation of the current simulations and model, as well as preliminary results, will be provided.

Speaker: Mr William O'Donnell (University of Glasgow)

MODE2024_WilliamO.pdf

10:00 → 10:40 Computer Science

Convener: Lisa Kusch (TU Eindhoven)

10:00 Charge Diffusion Modeling and Pixel-level Differentiability in Monolithic Active Pixel Sensors

Accurately simulating the response of monolithic active pixel sensors requires detailed technology computer-aided design simulations of the electric field inside the chip. This is used to model the electron propagation from their point of origin to potential collection. Specialized simulation software, such as Allpix², has been developed for this purpose. However, the electric field is often unobtainable due to the confidential nature of the chip's exact design properties. Further, such simulations are very time-consuming and difficult to differentiate, making them unappealing for end-to-end optimization pipelines.

In contrast, several analytical models for the charge collection process have been introduced, which do not require information about the electric field, but instead can be employed by fitting their parameters to match experimental data. We present a comparison of different approaches in terms of accuracy as well as differentiability of the models using data from an experiment with a small pixel telescope based on the ALPIDE sensor.

Speaker: Alexander Schilling (University of Kaiserslautern-Landau (RPTU))

Schilling_MODE_Workshop_2024_Differentiable_Charge_Diffusion_Modeling.pdf

10:20 Development of a Universal Simulation and Modelling Language at NASA

This presentation will describe a NASA project called the Universal Simulation and Modelling Language (USML) that is used as the computational engine for a mission called the Active Learning Physics Simulator (ALPS).

Background:
When performing physical simulations, there is a tradeoff between accuracy and computation time. For example, atomic-scale simulations are highly accurate but may be computationally intractable for larger lengthscales. Models must be hand-designed by scientists, and the accuracy of a model outside of its specific regime is uncertain. This is a particular problem for neural networks. Additionally, the underlying physics of a system may not be fully known, meaning that a model should be able to adapt itself to new information. We have addressed this problem through differentiable graph-based modelling.

Methodology:
The Minimum Description Length principle states that best explanation for a given set of data is the shortest description of that data. For some system described by model M, we can postulate an optimum graph-based language G that minimizes the description length of M in G. In a sense, this is a mathematical description of human intuition. Any Turing-complete programming language can be formulated into a graph notation where the nodes represent collections of properties and functions and the edges represent communication between nodes. By constructing a graph-based language for simulation, differentiability occurs as a natural consequence.

Results:
Early results of this project show promising versatility when applied to different domains. Tests on both atomic simulations and fluid-dynamics simulations produce a series of optimized models that range from fast-and-approximate to complex and highly accurate. This is accomplished by three key mechanisms: 1. Differentiability of graph objects allows for fast optimization, 2. The graph adapts to new information by expanding into higher dimensions and recompressing by removing edges through annealing, and 3. The uncertainty in each element of the graph is easy to measure, which allows for intelligent sampling of new information based on a model’s greatest areas of uncertainty. Item 3 in particular leads systematically to the Optimum Design of Experiments (ODE).

Conclusion:
Because the USML is abstracted away from everything except information theory and graph theory, the technology is truly domain-agnostic. In the future, we plan to use this technology both to accelerate scientific discoveries and to generate optimized engineering designs within the NASA agency. Additionally, there are multiple applications in various fields of particle physics. By presenting this work at the MODE workshop, we hope to share and improve this technology through communication with international collaborators.

Speaker: Stephen Casey (NASA Kennedy Space Center)

MODE_Casey.pptx

10:40 → 11:00 Astroparticle

Convener: Roberto Ruiz De Austri

10:40 ML4GW: An AI-based Ecosystem for Real-time Gravitational Wave Analysis

Deep learning algorithms have excelled in various domains. Despite this success, few deep-learning models have seen full end-to-end deployment in gravitational-wave searches, both in real-time and on archival data. In particular, there is a lack of standardized software tools for quick implementation and development of novel AI ideas. We address this gap by developing the ML4GW and HERMES libraries. We show how these libraries enhance efficiency and AI model robustness in the context of a broad range of gravitational wave analyses with an emphasis on real-time application, scalability to heterogeneous computing resources, and streamlining the training to deployment cycles for machine learning models. Building on this toolkit, we introduce specific machine learning pipelines that are currently running or set to run in real-time during the ongoing LIGO-Virgo-KAGRA observing run. These searches include Aframe, a low-latency machine learning pipeline for compact binary sources of gravitational waves; DeepClean, a deep learning-based denoising scheme for astrophysical gravitational waves; AMPLFI, a pipeline for deep learning based parameter estimation using likelihood-free inference; and GWAK, a semi-supervised strategy to identify unmodeled gravitational wave transients using embedded spaces. In addition, we discuss how such pipelines can lead to latency improvements for multi-messenger targets. Finally, we show how ML4GW and HERMES can quickly integrate the plethora of deep learning based algorithms being developed for gravitational wave identification across the broader astrophysics community.

Speaker: Eric Anton Moreno (Massachusetts Institute of Technology (US))

ML4GW_Mode_EMoreno.pdf

11:00 → 11:30 Coffee Break

11:30 → 12:30 Computer Science

Convener: Lisa Kusch (TU Eindhoven)

11:30 Automatic Differentiation in RooFit

With the growing datasets of HEP experiments, statistical analysis becomes more computationally demanding, requiring improvements in existing statistical analysis software. One way forward is to use Automatic Differentiation (AD) in likelihood fitting, which is often done with RooFit (a toolkit that is part of ROOT.) As of recently, RooFit can generate the gradient code for a given likelihood function with Clad, a compiler-based AD tool. At the CHEP 2023, and ICHEP 2024 conferences, we showed how using this analytical gradient significantly speeds up the minimization of simple likelihoods. This talk will present the current state of AD in RooFit. One highlight is that it now supports more complex models like template histogram stacks ("HistFactory"). It also uses a new version of Clad that contains several improvements tailored to the RooFit use case. This contribution will furthermore demo complete RooFit workflows that benefit from the improved performance with AD, such as CMS and ATLAS Higgs measurements.

Speaker: Vassil Vasilev (Princeton University (US))

CladRooFit.pdf

CladRooFit.pptx

11:50 Advanced optimizations for source transformation based automatic differentiation

Advanced optimizations for source transformation based automatic differentiation

Clad is a LLVM/Clang plugin designed to provide automatic differentiation (AD) for C++ mathematical functions. It generates code for computing derivatives modifying abstract syntax tree using LLVM compiler features. Clad supports forward- and reverse-mode differentiation that are effectively used to integrate all kinds of functions. The typical AD approach in Machine Learning tools records and flattens the compute graph at runtime, whereas Clad can perform more advanced optimizations at compile time using a rich program representation provided by the Clang AST. These optimizations investigate which parts of the computation graph are relevant to the AD rules. One such technique is the “To-Be-Recorded” optimization, which reduces the memory pressure to the clad tape data structure in the adjoint mode. Another optimization technique is activity analysis, which discards all derivative statements that are not relevant to the generated code. In the talk we will explain compiler-level optimizations specific to AD, and will show some specific examples of how these analyses have impacted clad applications.

Speaker: Maksym Andriichuk

mode_final_Maksym.pptx

12:10 Automatic Differentiation of the Kokkos framework with Clad

Kokkos is a high-performance library allowing scientists to develop performance-portable C++ code capable of running on CPUs, GPUs and exotic hardware. The Kokkos infrastructure enables researchers to write generic code for libraries, frameworks, and scientific simulations such as climate simulation tools like Albany and HOMMEXX that can later be run on a large scale on any supercomputing hardware without code rewrites.

Kokkos enables differentiable programming using operator overloading tool, Sacado, which records and executes the linearised computation graph. On the other side of the tool spectrum is Clad. It uses the source transformation approach to AD where more advanced optimisations can be investigated. For Kokkos, Clad brings reverse mode support and increased scalability. The challenge with source transformation tools is incorporating framework-specific knowledge and expressing the analytical primitives specific to the framework.

In this talk, we discuss how Clad works and enables AD for large domain-specific frameworks such as Kokkos. We describe how Clad handles support for the C++ STL as another example of its flexibility. We explain extension points such as user-defined custom derivatives, which allow derivatives of Kokkos constructs to be expressed in terms of themselves, without falling back to precise hardware-dependent definitions. We delve into the specifics of the process and lessons learned while integrating Clad with Kokkos and show results demonstrating how Clad has facilitated efficient and scalable automatic differentiation with Kokkos.

Speaker: Atell-Yehor Krasnopolski

MODE-Clad-for-STL-and-kokkos.pdf

12:30 → 12:40 Poster Awards

Speakers: Christian Glaser (Uppsala University), Emma Torro Pastor (Univ. of Valencia and CSIC (ES)), Lisa Kusch (TU Eindhoven)

PosterAwardCertificates2024.pdf

12:40 → 14:20 Lunch break

14:20 → 15:40 Medical Physics

Convener: Gabriela Llosá (Instituto de Física Corpuscular (IFIC) (CSCI-UV))

14:20 PETAL-3D: A Novel 3D U-Net Approach for Ultra-Low-Dose PET Image Denoising

Positron Emission Tomography (PET) is a functional imaging technique in nuclear medicine in which a radioactive tracer is injected into the patient to examine metabolic and physiological processes. Reducing the radiation dose to the patient is desirable and can be achieved by administering lower amounts of radiotracer. However, low-dose examinations result in increased noise level in the reconstructed images. To address this challenge, we have proposed a novel approach, Progressive Elimination of Noise Towards Accurate Ultra Low-Dose PET Images Using 3D U-Net (PETAL-3D). PETAL-3D iteratively improves the image quality of low-dose images by leveraging several sequential 3D U-Nets. PETAL-3D was trained on datasets of full and lowered dose scans from two state-of-the-art total-body PET scanners: Siemens Biograph Vision Quadra and United Imaging uEXPLORER. PETAL-3D's performance was compared against a 4-mm Gaussian filter, a traditional 3D U-Net, and an anatomically guided 3D U-Net. The results demonstrated that PETAL-3D significantly enhances image quality, as evidenced by global and local metrics, without introducing artifacts. This study underscores the potential of integrating deep learning in medical physics to optimize ultra-low-dose PET imaging. Since the application is done at the image level, it can be translated to other imaging systems and modalities.

Speaker: Mr Ezzat Elmoujarkach (Universität zu Lübeck - Institute of Medical Engineering)

MODE2024_Elmoujarkach.pptx

14:40 Neural Networks for inter-crystal scatter recovery in semi-monolithic PET detectors

Objective: One of the mayor challenges in positron emission tomography (PET) is to increase system efficiency without sacrificing spatial resolution. Including the contribution of inter-crystal scatter (ICS) events during image reconstruction is one way of achieving this aim, provided a method for estimating the primary photon path in such events is available. The IRIS group (IFIC, Valencia) is investigating the application of a neural network (NN) which utilizes 3D positioning information and energy deposition inside the detector to identify the primary interaction in ICS events. This approach has been implemented on novel semi-monolithic PET detector geometries developed by i3M (Valencia). Such detectors consist of monolithic LYSO scintillation crystals segmented in one direction into sections called slabs, combining the benefits of pixelated and continuous configurations, including 3D positioning capabilities.
Approach: Simulation studies have been carried out to test the performance of the NN for ICS recovery in two different detector designs: the IMAS total body PET, which utilizes slabs of 3.12 mm in width, each coupled to a single row of eight silicon photomultipliers (SiPMs); as well as the brain-dedicated 4D-PET, which achieves the high resolution required for brain imaging by using thin slabs of 1.6 mm in width in a light-sharing design, whereby pairs of slabs are coupled to a single row of SiPMs. Both detectors were simulated using GATE v8.2 and 511 keV photons tracked through the scintillation crystals. Singles were created on the slab level with experimental values for spatial and energy blurring
applied in post-processing. An energy window of 350-650 keV was used. Singles were grouped into single-photon events according to eventID and classified as either single-slab events, ICS events involving two slabs and ICS events involving three or more slabs. Only those events involving two slabs were considered for recovery. The NN used for ICS recovery was implemented in Python3 using Tensorflow and Keras. The data collected from the simulation consisted of the 3D spatial and energy information from each of the two singles comprising the single-photon event. The data was split into training, evaluation and test sets consisting of 90%, 5% and 5% of the data, respectively. The parameters used were depth of interaction (DOI), deposited energy, position along slab and separation between slabs. The NN architecture consisted of two hidden layers of 64 neurons each, with the ReLU activation function on the input and hidden nodes and the sigmoid activation function on the output node. The Adam optimizer was used for training as well as the binary crossentropy loss function. For the case of the light-sharing slab-pairs of the 4D-PET detector, a simulation that included optical photons was performed and a second NN, trained on light distribution on an 8x8 SiPM array, was implemented in order to distinguish ICS from photoelectric (PE) events in slab-pairs that share a common row of SiPMs. The 8x8 array was summed in both dimensions, yielding 1x16 input for the NN. Keras and Tensorflow were used, with the same parameters as above. The input layer and single hidden layer consisted of 16 neurons each.
Main results: The fraction of 511 keV photons interacting in two slabs, and therefore eligible for recovery using the NN, was found to be 26% and 29% for the IMAS and 4D-PET detectors, respectively. The respective fraction of single-slab events was 71% and 66%, with the remaining photons interacting in three or more slabs. The slab of first interaction was estimated in the 2-slab events with an accuracy 74% and 71% for the IMAS and 4D-PET detectors respectively. Identification of ICS (vs PE) in light-sharing detectors was performed with an accuracy of 98%.
Significance: This work presents a simulation study of two different semi-monolithic PET detectors, demonstrating that in such detectors a significant fraction of 511 keV photons undergo ICS in two slabs. The recovery of such events would significantly improve the scanner’s efficiency. The potential of using NNs and 3D positional and energy information for determining the slab of the first interaction in these ICS events has been demonstrated. Further work will concentrate on estimating 3D interaction positions and energy depositions in each of the slabs comprising a light-sharing pair in which ICS occurs, as the uncertainty on these quantities has been shown to impact the accuracy of estimating the
first interaction.

Speaker: Karol Brzezinski (IFIC (CSIC-UV))

Brzezinski_MODE2024.pdf

15:00 Machine Learning Framework for Time Pick-Up of Nuclear Detector Signals

Accurate timing characterization of radiation events is crucial in nuclear medicine, particularly for Positron Emission Tomography (PET). In PET, achieving a good coincidence resolving time (CRT) between detector pairs enhances the Time-of-Flight (TOF) information for each detected coincidence, which significantly improves the signal-to-noise ratio of the images. This study introduces a method to train models, based on the newly-developed Kolmogorov-Arnold networks (KANs), for assigning precise timestamps to incoming radiation signals in each detector. We trained the models with event pairs consisting of a measured event and its copy delayed a know amount of time where the delay acted as a label during training. Trained models were evaluated using data from a 60Co point source and a pair of conic 2” LaBr3(Ce) detectors in coincidence mode, connected to Hamamatsu R9779 PMTs sampled at 5 Gs/s. We report that our method has achieved a 6% increase in CTR and around 40% increased accuracy in source location compared to the widely used constant fraction discrimintation (CFD) method for the evaluation set.

Speaker: José Andrés Avellaneda González Not Supplied

Presentacion_MODE_Workshop_Javellan_2024.pdf

Presentacion_MODE_Workshop_Javellan_2024.pptx

15:20 Enhancing Compton Camera Imaging with Neural Networks

The application of neural networks in medical physics has shown significant promise in improving imaging techniques and treatment verification. The IRIS group of IFIC (Valencia) is an expert in developing Compton cameras for medical applications. The group employs neural networks to enhance the performance of such devices in different aspects. This work summarizes three key research studies where neural networks were employed to address challenges in Compton camera imaging.
The first study focuses on enhancing Compton camera imaging for multi-energy radioactive sources using neural networks for event selection. A Compton camera prototype (MACACO III) with monolithic LaBr3 crystals and silicon photomultiplier arrays was utilized to gather experimental data from a circular array of 22Na sources. The prototype and the circular array were simulated with Gate v8.0 code to obtain data for neural network training. The trained neural networks were used to classify simulated and
experimental data for selecting good events and rejecting background prior to image reconstruction. The trained models effectively selected useful events for the reconstruction algorithm and rejected background noise, resulting in significantly improved image quality compared to setting energy thresholds. This approach demonstrated the potential for neural networks to enhance the accuracy and quality of Compton camera imaging.
The second study aims to determine the photon interaction positions within a monolithic LaBr3 crystal using convolutional neural networks. Simulations were performed to generate training data for the neural network model. The light collected by the pixels of the photomultiplier array was recorded to generate images employed to train and test the model. The model's performance was evaluated based on full width at half maximum, Euclidean distance, and mean absolute error. The trained model demonstrated superior performance in predicting both 2D and 3D interaction positions, including the depth of interaction, compared to traditional analytical methods. This advancement is significant for improving the spatial resolution and accuracy of photon interaction positioning in medical imaging, leading to better diagnostic capabilities. The work is being extended to experimental data.
The third study explores a neural network-based method for signal event selection in Compton camera imaging, specifically for proton range verification in hadron therapy. Data were collected using the Compton camera prototype MACACO III at a clinical cyclotron facility and the same setup was simulated to train a neural network model for data selection. The neural network approach was compared to a traditional energy-cut method. Results indicated that the neural network method substantially increased the signal percentage and improved the precision in detecting millimetric shifts in depth-dose distribution. This enhancement is crucial for accurately monitoring and adjusting proton therapy treatments, ensuring better targeting of tumors and sparing healthy tissues.
These studies collectively highlight the transformative impact of neural networks in medical physics. By improving event selection in Compton camera imaging, accurately determining photon interaction positions and enhancing proton range verification, neural networks offer powerful tools for advancing medical imaging and treatment techniques. The findings presented underscore the broad applicability of machine learning in addressing complex problems in medical physics, paving the way for future innovations and improved patient care.

Speaker: Javier Pérez Curbelo (Instituto de Física Corpuscular (IFIC) (CSCI-UV))

MODE_Javier_Perez_Curbelo.pdf

15:40 → 16:10 Coffee Break

16:10 → 16:40 Introduction: Closing session

Convener: Dr Pietro Vischia (Universidad de Oviedo and Instituto de Ciencias y Tecnologías Espaciales de Asturias (ICTEA))

2024-09-25_MODEWorkshopClosing_vischia.pdf

HTML slides (preferred, animations work here)