Speaker
Description
Summary
The ATLAS detector's data acquisition system makes use of a multi-tiered trigger system to reduce bandwidth from the LHC proton crossing rate of 40 MHz to the 1kHz written to disk. The first tier (Level-1) is implemented in real time custom electronics and reduces the data flow to 100 kHz. The second and third tiers collectively referred to as the High Level Trigger (HLT) are implemented on PCs running custom triggering software. The software trigger is seeded by Regions of Interest (RoIs) identified in the Level-1 trigger hardware. These RoIs are provided by a custom VME system referred as the Region of Interest Builder (RoIB). The function of the RoIB is to collect raw event fragments from various L1 trigger sources, assemble all the fragments of a given event into an RoI record, and make this RoI record available to the HLT Supervisor (HLTSV) which manages the HLT processing farm.
The current RoIB is implemented on custom 9U VMEbus cards with 4 Input Cards that accommodate three S-Link inputs each and one Builder Card that sends data via two S-LINK outputs to the HLTSV. The receiver card in the HLSTV is a TILAR card that implements a 4 PCIe Gen1 lanes to interface the two S-LINK data channels. The custom VME based RoIB operated reliably during the first run of the LHC. However, with the increase of luminosity and the complexity of L1 triggers, it is desirable to have the RoIB more operationally maintainable, which will minimize the amount of custom hardware that needs to be supported for reliable ATLAS data taking. In this regard, a solution was devised to implement the construction of a RoI record at software level and thus bypassing the need for a custom RoIB hardware. The new system is required to achieve over 100 kHz readout rate over 12 input channels for fragment sizes of 128 32 bit words.
A custom board developed by the ALICE collaboration, Common ReadOut Receiver Card (C-RORC), is used in the ATLAS ROS buffers with an ATLAS specific firmware and software called RobinNP. The new software based RoIB uses the RobinNP framework to perform the RoIB functionality.
The C-RORC implements 8 PCIe Gen 1 lanes with 1.4 GB/s bandwidth to the CPU fed via 12 optical links each running 200 MB/s on 3 QSFP transceivers. It utilizes a single Xilinx Virtex-6 series FPGA that handles data input from the 12 optical links and buffers the data in the on-board DDR3 memory. It is also capable of processing and initiating DMA transfer of event data from the on-board memory to the PC memory.
Tests were performed to evaluate the input/output bandwidth of the RobinNP using a high performance processor (Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.5GHz). The preliminary measurements show that the new RoIB will achieve a rate of 150 kHz for fragments with a size of 128 32 bit words.
