Speaker
Description
During the current major LHC shutdown (2019-2021), the ATLAS experiment at CERN is moving to the Front-End Link eXchange (FELIX) system as the interface between the data acquisition system and the trigger and detector front-end electronics. FELIX functions as a router between custom serial links and a commodity switch network, which uses industry-standard technologies to communicate with data collection and processing components. This presentation will describe the FELIX system design as well as report on the installation and commissioning of the full system in summer 2019.
Summary
After the current LHC shutdown (2019-2021), the ATLAS experiment will be required to operate in a significantly harsher collision environment. The LHC will deliver luminosities up to three times the original design value, with a commensurate increase in the number of interactions per bunch crossing. To maintain physics performance in this new regime, the ATLAS experiment will undergo a series of upgrades during the shutdown. Among the upgraded components will be the Trigger and Data Acquisition (TDAQ) system, which in the new environment will have to process significantly more complex events while maintaining selection performance. At the same time, the TDAQ system will have to interface with new on-detector readout technologies for the new Muon Small Wheel detector, the Liquid Argon (LAr) Calorimeter and Calorimeter Trigger upgrades. In total, the new readout paths will require approximately 2000 new optical links to be serviced by the TDAQ system.
The upgraded ATLAS systems will make use of newer readout link technologies. These include the high bandwidth FULL mode ˜FPGA-to-FPGA protocol, running at 10 Gb/s, and the radiation-hard Giga Bit Transceiver (GBT) protocol, running at 5 Gb/s. GBT links provide aggregation for up to 42 slower serial electrical links, making them critical for data collection from the front-end ASICs within the new systems. GBT links will also be used for the propagation of control and configuration data back to the front-end. To connect the new systems, and handle the significantly increased data volumes in a detector agnostic and easily scalable way, a new readout architecture named the Front-End LInk eXchange (FELIX) has been developed.
FELIX receives and identifies different information streams on its incoming optical links and routes packets to client processing applications via a commercial switched network. In the opposite direction, FELIX receives packets from the network and forwards them to specific on-detector modules. FELIX also handles input from the Timing, Trigger and Control (TTC) system to recover the LHC clock and forward synchronous trigger information to on-detector electronics over low-and-fixed-latency GBT links. FELIX supports multiple different data encoding formats, including 8b/10b and HDLC, to satisfy the requirements both of primary dataflow and experimental slow control.
The final implementation of FELIX makes use of a custom built PCIe board with a Xilinx Kintex UltraScale FPGA, a 16 lane Gen3 PCIe (128 Gb/s) interface and 48 bidirectional optical interfaces in the form of eight Mini-POD transceivers (max. link speed 14 Gb/s). TTC decoding circuitry is hosted on a custom mezzanine board to enable future upgrades of the TTC system to be supported with minimal hardware changes. The optical links, PCIe interface, and TTC decoding circuits have been verified to function well in the final hardware.
The FELIX system is due to be installed and commissioned in summer 2019. This presentation will report on the status of this installation, and the results of ongoing commissioning work.
