Fast Machine Learning for Science Conference 2026

An extensible transformer architecture implementation on AMD AI-engines for ultra-low-latency stream-data processing

Not scheduled

12m

Presentation Contributed Talks

Speakers

Elias Leutgeb (CERN) Fabian Helmberger (Vienna University of Technology (AT)) Thomas Owen James (CERN)

Description

Deep learning techniques are being used for event classification, reconstruction, and triggering in the first stages of data selection and processing at collider experiments, such as CMS and ATLAS at the LHC. Thus far, the networks being used for low-latency inference are predominately based on fully-connected architectures. On the other hand, transformer-based ML models have demonstrated significant advantages over conventional neural network architectures in a wide range of machine-learning applications, and are now being developed for use in high energy physics. This project aims to leverage the AMD Versal AI Engines in order to implement a configurable and flexible transformer architecture on these devices, capable of stream-processing large amounts of detector data, optimized for a latency on the order of microseconds.

The implementation has been developed using the AMD Vitis toolchain for deployment on the VEK280 development platform, which hosts a second generation AMD Versal device, the AI Edge Series VE2802.

We present a resource-aware fixed-point implementation of multi-head attention specialized for the tile-level programmability of the AI Engine array. The resulting throughput and latency are evaluated for transformer models of varying sizes, and strategies for scaling the number of attention heads, the embedding dimension, and the number of input tokens are discussed.

Finally, we outline potential applications of this implementation to particle physics experiments, including the CMS Level-1 Scouting system, where latency constraints are less stringent than those of the primary trigger path.