Speaker
Description
Real-time identification of jets from boosted heavy particles is vital for the High-Luminosity LHC physics program at CMS. However, deploying highly expressive architectures like the Particle Transformer within the hardware-constrained Level-1 (L1) trigger is severely limited by the quadratic scaling $\mathcal{O}(N^2)$ of standard self-attention. This computational burden makes microsecond-scale execution on Field Programmable Gate Arrays (FPGAs) resource-prohibitive. We present ErwinParT, a hierarchical transformer architecture designed for real-time edge deployment through a physics-informed structural partitioning scheme optimized for maximum hardware efficiency.
ErwinParT leverages the conical topology of high-energy jets by sorting constituent particles by their angular separation $\Delta R$ relative to the jet axis. This sequencing preserves spatial locality in the initial index mapping. The sorted constituents are then clustered into localized, fixed-size partitions ("balls"). Computing multi-head self-attention exclusively within these small neighborhoods mitigates the quadratic scaling bottleneck through an $\mathcal{O}(N \cdot B)$ formulation, where B is the bounded partition size. To retain the fine-grained substructure features needed for robust jet discrimination, the model incorporates Relative Position Embeddings (RPE) based on coordinate offsets from each partition’s center of mass, along with a distance-based attention bias that enforces continuous spatial decay.
Global context is recovered across the entire jet through an alternating cross-ball interaction mechanism akin to shifted window configurations. Systematically shifting the partition boundaries between consecutive transformer layers allows adjacent radial zones to exchange features. This establishes a global receptive field over the full depth of the network while bypassing the information bottlenecks typical of aggressive down-sampling architectures.
By design, this optimized $\mathcal{O}(N \cdot B)$ formulation is structured to map naturally onto the hardware layer of CMS L1 trigger FPGAs as parallel, deterministic compute loops. Replacing all-to-all quadratic attention matrices with segmented, fixed-size matrix blocks is expected to significantly reduce Digital Signal Processor (DSP) and Block RAM (BRAM) utilization. This architectural alignment aims to yield a predictable, low-clock-cycle pipeline depth, making high-fidelity transformer inference a viable candidate for the sub-microsecond real-time data acquisition stream of the CMS experiment.
| Do you plan to submit a 4-page extended abstract on OpenReview (only for Presentations/Posters)? | Maybe |
|---|