Sep 26 – 30, 2016
Karlsruhe Institute of Technology (KIT)
Europe/Zurich timezone

Evaluation of GPUs for High-Level Triggers in High Energy Physics

Sep 29, 2016, 3:40 PM
Redtenbacher Lecture Hall (Building 10.91)

Redtenbacher Lecture Hall (Building 10.91)

Oral Trigger Trigger


Hannes Heiner Mohr (KIT - Karlsruhe Institute of Technology (DE))


Modern High Energy Physics Trigger Systems require high data throughput on the Gigabyte scale and latencies in the range of a few microseconds.
Traditionally, those requirements could only be met by expensive, dedicated hardware like FPGAs and ASICs.
However, GPUs provide high-performance and pose an affordable and easily programmable alternative.
In this paper we evaluate modern GPGPUs as a flexible alternative to traditional approaches.
We discus the performance, throughput and latency of commonly used algorithms and give an overview of possible benefits as well as downsides of this approach.
Finally, we give a brief outlook of possible future developments.


The CMS experiment at the LHC at CERN will face a major luminosity increase in its upcoming phase 2 upgrade.
With a luminosity of $10^{34}\,\mathrm{cm}^{-2}\mathrm{s}^{-1}$ corresponding to an increase in the number of collsions by a factor of 10 and pileups of around 140.
The detector produces roughly $50\,\mathrm{Tb}/\mathrm{s}$ of tracker data that need to be processed by the trigger systems within a timing window of as low as $12\,\mu\,\mathrm{s}$.
In order to meet such requirements, new trigger architectures need to be developed. Such systems are commonly developed based on FPGAs and ASICs, which, while being highly customizable for the application, are generally expensive, both in cost and maintenance time.

The Institute for Data Processing and Electronics at the Karlsruhe Institute of Technology has started to investigate alternative approaches for such trigger systems, based on modern high-performance computing (HPC) technologies.

One of the most promising trends in the area of HPC is the processing of data with the aid of general purpose graphics processing units (GPGPUs). GPGPUs have seen rapid advances in performance over the last decade.
Software development frameworks such as CUDA and OpenCL have further increased the accessibility of GPGPUs for scientific computing.

In this work we evaluate Track Seeding and Track Finding approaches, based on GPGPUs to be used as online Level-1 Track Trigger Systems at the example of the CMS experiment, specific to it's phase 2 upgrade.

We compare the performance and latency on different generations of graphics card, both on older and very recent hardware, as well as for both prominent Software Development Frameworks, OpenCL and CUDA, and discuss their respective benefits and limitations.

First results show processing times as low as 7 microseconds for the track seeding step, utilizing a Hough Transform approach, with a detector segmentation similar to what has been used by FPGA based trigger systems.
We present performance measurements for track seeding and track fitting and discuss their respective suitability for GPUs in general, as well as their flexibility with regard to possible changes in the not yet fully realized detector layout.
Furthermore, we also briefly discuss the quality of the used algorithms in terms of track finding efficiency, based on the official, preliminary Monte Carlo Simulation data for the detector. The results are validated by the CMS Software Framwork (CMSSW).

An overview of recent graphics processors and memory developments is given, to provide an estimate of the performance that we may expect to see in the near future.

The results so far indicate that GPUs are a potential candidate for future trigger systems, while simultaneously offering more complex computational capabilities for the ever increasing requirements of HEP experiments.

Primary authors

Hannes Heiner Mohr (KIT - Karlsruhe Institute of Technology (DE)) Timo Dritschler (KIT)


Dr Andreas Kopmann (KIT) Lorenzo Rota (KIT) Luis E. Ardila Marc Weber (KIT - Karlsruhe Institute of Technology (DE)) Matthias Norbert Balzer (KIT - Karlsruhe Institute of Technology (DE)) Matthias Vogelgesang (Karlsruhe Institute of Technology) Dr Michele Caselle (Karlsruhe Institute of Technology) Suren Chilingaryan (KIT) Thomas Schuh (KIT - Karlsruhe Institute of Technology (DE))

Presentation materials