Mini-workshop on Real time tracking : triggering events with tracks

Friday 13 Oct 2023, 09:00 → 18:00 Europe/Zurich

Auditorium (Le Village)

Alexis Vallier (L2I Toulouse, CNRS/IN2P3, UT3), Mark Neubauer (Univ. Illinois at Urbana Champaign (US)), Vladimir Gligorov (Centre National de la Recherche Scientifique (FR))

High-Energy physics experiments operating at the large hadron collider must process tens of terabytes of data per second, rising to hundreds over the next decade. Accurate, precise, and reliable real-time reconstruction of charged particle trajectories (tracks) is increasingly essential so that experiments may rapidly select a small fraction of this data which is of interest to physics analysis and discard the rest. Over the past decade in particular numerous technological solutions exploiting different highly parallel computing architectrues have emerged which allow tracking to be performed at ever earlier stages in the real-time processing pipeline, enhancing the physics reach of experiments.

The scope of this one day mini-workshop following the Connecting The Dots 2023 conference is to discuss the existing and planned technological solutions to be able to reconstruct particle tracks and vertices in real-time, either in a triggerless system with a full online event reconstruction or within the trigger system. Algorithmic principles of parallel pattern recognition and portability on specific architectures (CPU, GPU, FPGA…) will be covered, with an outlook to the experiments and facilities of the next decades. 

1 Directions in Realtime Tracking : Everything, Everywhere, All at Once

Speaker: Kristian Hahn (Northwestern University (US))

Realtime_tracking_Hahn.pdf

2 Level-1 Tracking at CMS for the HL-LHC

The success of the CMS physics program at the HL-LHC requires maintaining sufficiently low trigger thresholds to select processes at the electroweak scale. With an average expected 200 pileup interactions, critical to achieve this goal while maintaining manageable trigger rates is in the inclusion of tracking in the L1 trigger. A 40 MHz silicon-tracker based track trigger on the scale of the CMS detector has never before been built; it is a novel handle, which in addition to maintaining trigger rates can enable entirely new physics studies.

The main challenges of reconstructing tracks in the L1 trigger are the large data throughput at 40 MHz and the need for a trigger decision within 12.5 µs out of which 4 µs is for track finding. The CMS outer tracker for HL-LHC uses modules with closely-spaced silicon sensors to read out only hits compatible with charged particles with pT above 2 GeV ("stubs"). These are used in the backend L1 track finding system, based on commercially available FPGA technology. The ever-increasing capability of modern FPGAs combined with their programming flexibility is ideal for a fast track finding algorithm. The L1 tracking algorithm forms track seeds ("tracklets") from pairs of stubs in adjacent layers of the outer tracker. These seeds provide roads where consistent stubs are included to form track candidates. Track candidates sharing multiple stubs are combined prior to being fitted. A Kalman Filter track fitting algorithm is employed to identify the final track candidates and determine the track parameters. The system is divided into nine sectors in the r-phi plane, where the processing for each sector is performed by a dedicated track finding board.

This presentation will discuss the latest status of the CMS L1 track finding and its implementation, present simulation studies of the estimated performance, and discuss the developments of hardware demonstrators.

Speaker: Sara Fiorendi (University of Tennessee (US))

L1TF_CMS_CTD2023.pdf

3 Standalone track reconstruction and matching algorithms for the GPU-based High Level Trigger at LHCb

The LHCb Upgrade in Run 3 has changed its trigger scheme for a full software selection in two steps. The first step, HLT1, will be entirely implemented on GPUs and run a fast selection aiming at reducing the visible collision rate from 30 MHz to 1 MHz.
This selection relies on a partial reconstruction of the event. A version of this reconstruction starts with two monolithic tracking algorithms, the VELO-pixel tracking and the HybridSeeding on Scintillating-Fiber tracker, which reconstructs track segments in standalone subdetectors. Those segments are then matched through a matching algorithm in order to produce ‘long’ tracks, which form the base of the HLT1 reconstruction. We discuss the principle of these algorithms as well as the details of their implementation which allows them to run at a high-throughput configuration. An emphasis is put on the optimizations of the algorithms themselves in order to take advantage of the GPU architecture. Finally, results are presented in the context of the LHCb performance requirements for Run 3.

Speaker: Louis Henry (EPFL - Ecole Polytechnique Federale Lausanne (CH))

20231013_LHenry_LHCb_GPU.pdf

4 FPGA-based architecture for a real-time track reconstruction in the LHCb Scintillating Fibre Tracker beyond Run 3

Finding track segments downstream of the magnet is some of the most important and computationally expensive task of the first stage of the new GPU-based software trigger of the LHCb Upgrade I, that has started operation in Run 3. These segments are essential to form all good physics tracks with a very high precision momentum measurement, when combined with those reconstructed in the vertex track detector, and to reconstruct long-lived particles, such as Kshort and strange baryons, decaying after the vertex track detector, largely boosting the physics reach of the experiment. In this talk, we discuss the collaboration plans to install a real-time tracking device based on distributed system of FPGAs, dedicated to the reconstruction of particles trajectories in the forward Scintillating Fibre tracker detector, with the aim to preserve the full physics potential of the experiment in Run 4, and in view of Run 5 (Upgrade II) at higher instantaneous luminosity. This system will enhance the DAQ system of the experiment, and will run in real time during physics data taking, reconstructing tracks on-the-fly at the LHC collision rate, before the software trigger processing begins. The design and the expected performance of this device, with the capability of processing events at the full LHC collision rate of 30 MHz is discussed.

Speaker: Michael J. Morello (SNS and INFN-Pisa (IT))

2023_CTDTs_v1.pdf

10:40 Coffee

5 A real-time demonstrator of track reconstruction with FPGAs at LHCb

The upgraded LHCb detector has started its Run 3 of data taking in 2022, with a completely overhauled DAQ system, reading out and processing the full detector data at every LHC bunch crossing (30 MHz average rate). At the same, an intense R&D activity is taking place, with the aim of further improving the real-time data processing performance of LHCb, in view of a further luminosity upgrade of the experiment (“Upgrade II”).
In this work, we describe the experience gained with a prototype device for a 30 MHz real-time tracking in the LHCb VELO detector, implemented in state-of-art PCIe-hosted FPGA cards interconnected by fast optical links.
The system is capable of processing live LHCb data opportunistically during physics data taking, thanks to a dedicated testbed facility fed by the experiment monitoring system. We describe, amongst other things, the system used to organize and optimize the high-speed distribution of data to the components, and the synchronization with the most updated alignment constants to be used in track reconstruction.

Speaker: Francesco Terzuoli (Università di Siena & INFN Pisa (IT))

[Terzuoli]CDT2023_RETINA_Real_Time_Demonstrator_at_LHCb.pdf

6 Trigger Level Tracking With Neural Networks on Heterogeneous Computing Systems

The high luminosity upgrade of the LHC aims to better probe the higgs potential and self coupling. The Event Filter task force has been charged with exploring novel approaches to charged particle tracking to be employed in the upgraded ATLAS trigger system, capable of analyzing high luminosity events in real time. We present a neural network (NN) based approach to predicting and identifying hits left by particles at trigger level. In this bottom-up approach, the complexity of and input to the NN are kept minimal to allow the NN to be implemented in a heterogeneous computing system, such as with FPGA or GPU. This hardware based approach allows for increased data throughput, shorter latency and, crucially, flexibility to improve the algorithm in the future.

Speaker: Alex Gekow (Ohio State University (US))

Connecting The Dots 2023 (9).pdf

7 Track reconstruction for the ATLAS Phase-II High-Level Trigger using Graph Neural Networks on FPGAs

The High-Luminosity LHC (HL-LHC) will provide an order of magnitude increase in integrated luminosity and enhance the discovery reach for new phenomena. The increased pile-up foreseen during the HL-LHC necessitates major upgrades to the ATLAS detector and trigger. The Phase-II trigger will consist of two levels, a hardware-based Level-0 trigger and an Event Filter (EF) with tracking capabilities. Within the Trigger and Data Acquisition group, a heterogeneous computing farm consisting of CPUs and potentially GPUs and/or FPGAs is under study, together with the use of modern machine learning algorithms such as Graph Neural Networks (GNNs).

GNNs are a powerful class of geometric deep learning methods for modeling spatial dependencies via message passing over graphs. They are well-suited for track reconstruction tasks by learning on an expressive structured graph representation of hit data and considerable speedup over CPU-based execution is possible on FPGAs.

The focus of this talk is a study of track reconstruction for the Phase-II EF system using GNNs on FPGAs. We explore each of the steps in a GNN-based EF tracking pipeline: graph construction, edge classification using an interaction network (IN), and track reconstruction. Several methods and hardware platforms are under evaluation, studying optimizations of the GNN approach aimed to minimize FPGA resources utilization and maximize throughput while retaining high track reconstruction efficiency and low fake rates required for the ATLAS Phase-II EF tracking system. These studies include IN model hyperparameter tuning, model pruning and quantization-aware training, and sequential processing of sub-graphs over the detector.

Speaker: Sachin Gupta (Ruprecht Karls Universitaet Heidelberg (DE))

Satellite_CTD_SachinGupta.pdf

8 Studies on track finding algorithms based on machine learning with GPU and FPGA

Track finding in high-density environments is a key challenge for experiments at modern accelerators. In this presentation we describe the performance obtained running machine learning models studied for the ATLAS Muon High Level Trigger. These models are designed for hit position reconstruction and track pattern recognition with a tracking detector, on a commercially available Xilinx FPGA: Alveo U50, Alveo U250, and Versal VCK5000. We compare the inference times obtained on a CPU, on a GPU and on the FPGA cards. These tests are done using TensorFlow libraries as well as the TensorRT framework, and software frameworks for AI-based applications acceleration. The inference times obtained are compared to the needs of present and future experiments at LHC.

Speaker: Francesco Armando Di Bello (INFN e Universita Genova (IT))

fast_inference_tolosa.pdf

12:40 Lunch