Speaker
Description
Transformers are the state-of-the-art model architectures and widely used in application areas of machine learning. However the performance of such architectures is less well explored in the ultra-low latency domains where deployment on FPGAs or ASICs is required. Such domains include the trigger and data acquisition systems of the LHC experiments.
We present a transformer-based algorithm for jet tagging built with the HGQ2 framework, which is able to produce a model with heterogeneous bitwidths for fast inference on FPGAs, as required in the trigger systems at the LHC experiments. The bitwidths are acquired during training by minimizing the total bit operations as an additional parameter. By allowing a bitwidth of zero, the model is pruned in-situ during training. Using this quantization-aware approach, our algorithm achieves state-of-the-art performance while also retaining permutation invariance which is a key property for particle physics applications
Due to the strength of transformers in representation learning, our work serves also as a stepping stone for the development of a larger foundation model for trigger applications.
Significance
Previous transformer implementations for FPGA inference do not reach good accuracy due to limited quantization aware training and thus the inability to use larger model size, which is required for good performance for transformers.
This work introduces novel high granularity quantization scheme for transformers, which enables the deployment of larger transformer models that reach state-of-the-art accuracy.
References
Gradient-based Automatic Mixed Precision Quantization for Neural Networks On-Chip https://arxiv.org/abs/2405.00645
Accelerating Transformer Neural Networks on FPGAs for High Energy Physics Experiments https://www.doc.ic.ac.uk/~wl/papers/22/fpt22fw.pdf
Fast Jet Tagging with MLP-Mixers on FPGAs https://arxiv.org/abs/2503.03103
Ultrafast jet classification on FPGAs for the HL-LHC https://arxiv.org/abs/2402.01876
