Speaker
Description
The ATLAS Phase-I upgrade requires a Trigger and Data Acquisition (TDAQ) system able to trigger and record data from up to three times the nominal LHC instantaneous luminosity. The FELIX system provides this in a scalable, detector agnostic and easily upgradeable way. It is a PC-based gateway, routing between custom radiation tolerant optical links from front-end electronics, via FPGA PCIe Gen3 cards, and a commodity switched Ethernet or InfiniBand network. FELIX enables reducing custom electronics in favor of software on commercial servers. The FELIX system, results of demonstrator, design and testing of prototype are described.
Summary
The ATLAS Phase-I upgrade requires a Trigger and Data Acquisition (TDAQ) system able to trigger and record data from up to three times the nominal LHC instantaneous luminosity. A new detector independent readout architecture, named
Front-End LInk eXchange (FELIX), provides this in a scalable, detector agnostic and easily upgradeable way. During LHC Long Shutdown 2 (2019-2020), new ATLAS on detector electronics for the Liquid Argon (LAr) Calorimeters and new muon detectors will be installed. Radiation-hard Giga Bit Transceivers (GBT) will be used for data transmission. By means of time multiplexing, the GBT protocol provides up to 42 independent logical data links, yet sharing the same fiber. Through such links FELIX receives and identifies different information streams. Subsequently data packets will be routed via a commercial switched network. In the opposite direction, FELIX receives packets from the network and forwards them to specific on-detector modules. Another task of FELIX is to handle input from the Time, Trigger and Control (TTC) system to recover the LHC clock and to forward the synchronous trigger information. This information will be distributed to on-detector electronics over low-and-fixed-latency GBT links, and also to new and upgraded off-detector first-level trigger systems. For readout of the latter a lightweight protocol with higher throughput than the GBT protocol is envisaged to be used.
The functions described above are implemented in FPGAs. The Hitech Global HTG-710 is used as demonstrator card: it is equipped with an 8 lanes PCIe Gen3 (64 Gb/s) interface and with two CXP transceivers providing interfaces for 24 bidirectional optical links (max. 13.1 Gb/s). Moreover a custom mezzanine was designed to receive and decode the TTC clock and data information. The firmware is also for Xilinx VC-709 evaluation board, which has same type of FPGA and PCIe interface as HTG-710, but less optical interfaces. This board targets detector and trigger system test setups.
Drivers and software tools have been developed for testing and configuration of the boards. Data routing and the connection to the COTS (Commercial Off-The-Shelf) network is implemented in a software pipeline running on the FELIX host PC. The packet processing performance satisfies the requirement of FELIX.
A PCIe board with a Xilinx Kintex UltraScale FPGA, a 16 lanes Gen3 PCIe interface, and 48 bidirectional optical interfaces in the form of eight Mini-POD transceivers (max. link speed 14 Gb/s) will probably be the baseline choice to be used in the Phase-I FELIX systems. The optical links, PCIe interface, and TTC decoding circuits of the first prototype have been verified to function well. Integration testing with the complete FELIX firmware and software is ongoing.
In this paper, the FELIX system will be introduced. Then the optimization to the fixed low-latency GBT core from CERN will be discussed. Furthermore results of the demonstrator testing, and progress of the prototype design and testing will be presented.
