Speaker
Description
Starting during the upcoming major LHC shutdown (2019-2021), the ATLAS experiment at CERN will move to the Front-End Link eXchange (FELIX) system as the interface between the data acquisition system and the trigger and detector front-end electronics. FELIX will function as a router between custom serial links and a commodity switch network, which will use industry standard technologies to communicate with data collection and processing components. This presentation will describe the FELIX system design as well as report on results of the ongoing development program.
Summary
After the upcoming LHC shutdown (2019-2020), the ATLAS experiment will be required to operate in a significantly harsher collision environment. The LHC will deliver luminosities up to three times the original design value, with a commensurate increase in the number of interactions per bunch crossing. To maintain physics performance in this new regime, the ATLAS experiment will undergo a series of upgrades during the 2019-2020 shutdown. Among the upgraded components will be the Trigger and Data Acquisition (TDAQ) system, which in the new environment will have to process significantly more complex events while maintaining selection performance. At the same time, the TDAQ system will have to interface with new on-detector readout technologies for the new Muon Small Wheel detector, the Liquid Argon (LAr) Calorimeter and Calorimeter Trigger upgrades. In total, the new systems will require
approximately 2000 new optical links to be serviced by the TDAQ system.
The upgraded ATLAS systems will make use of newer readout link technologies. These include the high bandwidth FULL mode ‘FPGA-to-FPGA’ protocol, running at 10 Gb/s, and the radiation-hard Giga Bit Transceiver (GBT) protocol, running at 5 Gb/s. GBT links provide aggregation for up to 42 slower serial electrical links, making them critical for data collection from the front-end ASICs within the new systems. GBT links will also be used for the propagation of control and configuration data back to the front-end. To connect the new systems, and handle the significantly increased data volumes in a detector agnostic and easily scalable way, a new readout architecture named the Front-End LInk eXchange (FELIX) is under development.
FELIX receives and identifies different information streams on its incoming optical links and routes packets to client processing applications via a commercial switched network. In the opposite
direction, FELIX receives packets from the network and forwards them to specific on-detector modules. FELIX also handles input from the Timing, Trigger and Control (TTC) system to recover the LHC clock and forward synchronous trigger information to on-detector electronics over low-and-fixed-latency GBT links. FELIX supports multiple different data encoding formats, including 8b/10b and HDLC, to satisfy the requirements both of primary dataflow and experimental slow control.
The final implementation of FELIX makes use of a custom built PCIe board with a Xilinx Kintex UltraScale FPGA, a 16 lane Gen3 PCIe (128 Gb/s) interface and 48 bidirectional optical interfaces in the form of eight Mini-POD transceivers (max. link speed 14 Gb/s). TTC decoding circuitry is hosted on a custom mezzanine board to enable future upgrades of the TTC system to be supported with minimal hardware changes. The optical links, PCIe interface, and TTC decoding circuits have been verified to function well in all prototypes.
Integration testing with the final prototype and the complete FELIX firmware and software infrastructure is ongoing.
The results of these tests will be presented in this contribution, showing that the packet processing performance of the design satisfies ATLAS readout requirements for 2021.
