28 September 2015 to 2 October 2015
Lisbon
Europe/Zurich timezone

The Evolution of the Region of Interest Builder in the ATLAS experiment

29 Sep 2015, 17:29
1m
Hall of Civil Engineering (Lisbon)

Hall of Civil Engineering

Lisbon

IST (Instituto Superior Técnico ) Alameda Campus Av. Rovisco Pais, 1 1049-001 Lisboa Portugal
Poster Trigger Poster

Speaker

Othmane Rifki (University of Oklahoma (US))

Description

The ATLAS readout architecture uses the concept of Regions of Interest (RoIs) identified by the hardware trigger for further analysis in trigger software. The High Level Trigger uses these RoIs to guide the retrieval of information from the readout system. Currently, a custom VME system, the RoI Builder (RoIB), collects the RoIs at 100 kHz. This talk describes the evolution of the RoIB to a PCIe card, the C-RORC, in a PC running dedicated software. The C-RORC implements 8 PCIe Gen 1 lanes with 12 optical links each running at 200 MB/s on 3 QSFP transceivers.

Summary

The ATLAS detector's data acquisition system makes use of a multi-tiered trigger system to reduce bandwidth from the LHC proton crossing rate of 40 MHz to the 1kHz written to disk. The first tier (Level-1) is implemented in real time custom electronics and reduces the data flow to 100 kHz. The second and third tiers collectively referred to as the High Level Trigger (HLT) are implemented on PCs running custom triggering software. The software trigger is seeded by Regions of Interest (RoIs) identified in the Level-1 trigger hardware. These RoIs are provided by a custom VME system referred as the Region of Interest Builder (RoIB). The function of the RoIB is to collect raw event fragments from various L1 trigger sources, assemble all the fragments of a given event into an RoI record, and make this RoI record available to the HLT Supervisor (HLTSV) which manages the HLT processing farm.
The current RoIB is implemented on custom 9U VMEbus cards with 4 Input Cards that accommodate three S-Link inputs each and one Builder Card that sends data via two S-LINK outputs to the HLTSV. The receiver card in the HLSTV is a TILAR card that implements a 4 PCIe Gen1 lanes to interface the two S-LINK data channels. The custom VME based RoIB operated reliably during the first run of the LHC. However, with the increase of luminosity and the complexity of L1 triggers, it is desirable to have the RoIB more operationally maintainable, which will minimize the amount of custom hardware that needs to be supported for reliable ATLAS data taking. In this regard, a solution was devised to implement the construction of a RoI record at software level and thus bypassing the need for a custom RoIB hardware. The new system is required to achieve over 100 kHz readout rate over 12 input channels for fragment sizes of 128 32 bit words.
A custom board developed by the ALICE collaboration, Common ReadOut Receiver Card (C-RORC), is used in the ATLAS ROS buffers with an ATLAS specific firmware and software called RobinNP. The new software based RoIB uses the RobinNP framework to perform the RoIB functionality.
The C-RORC implements 8 PCIe Gen 1 lanes with 1.4 GB/s bandwidth to the CPU fed via 12 optical links each running 200 MB/s on 3 QSFP transceivers. It utilizes a single Xilinx Virtex-6 series FPGA that handles data input from the 12 optical links and buffers the data in the on-board DDR3 memory. It is also capable of processing and initiating DMA transfer of event data from the on-board memory to the PC memory.
Tests were performed to evaluate the input/output bandwidth of the RobinNP using a high performance processor (Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.5GHz). The preliminary measurements show that the new RoIB will achieve a rate of 150 kHz for fragments with a size of 128 32 bit words.

Primary author

Othmane Rifki (University of Oklahoma (US))

Co-authors

Barry Green (University of London (GB)) Gordon Crone (University College London (UK)) James Proudfoot (Argonne National Laboratory (US)) Jeremy Robert Love (Boston University (US)) Jinlong Zhang (Argonne National Laboratory (US)) Jose Guillermo Panduro Vazquez (Royal Holloway, University of London) Robert Blair (Argonne National Laboratory (US))

Presentation Materials