Speaker
Description
The FrontEnd LInk eXchange (FELIX) is an FPGA-based data router designed to interface custom detector readout systems, and commodity switched networks as part of the ongoing upgrade of the ATLAS experiment at CERN. FELIX relies on synchronous data aggregation with GBT and lpGBT protocols to control and readout multiple detector front-ends. To facilitate validation and benchmarking, we designed and used an FPGA-based emulator of the front-end systems, FELIG. FELIG uses the same hardware as FELIX, FLX-712 board and inherits selected firmware blocks from FELIX. However, FELIG features clock and data recovery and a configurable data generator specifically designed for it.
Summary (500 words)
The FELIG system emulates detector front-ends to verify and benchmark the FELIX system. FELIG can use GBT or lpGBT protocols and generate output data. The output data can be 8b/10b encoded, and the content is configurable. FELIG uses the FLX712 hardware platform, just like FELIX. FELIX and FELIG are interconnected with fiber-optical data links, and FELIG recovers the 40.08 MHz clock from the incoming bitstream.
FELIG was first designed for the commercially available HTG-710 board, and more recently, ported to the latest version of the FELIX custom hardware platforms, the FLX-712. The FLX-712 hosts a Xilinx Kintex UltraScale FPGA and supports 48 high-speed optical links via minipods. Communication with the host computer is performed using a 16 lane PCIe Gen 3 interface connected to two PCIe hardcore EndPoints in the FPGA.
FELIG inherits from FELIX firmware design. A block diagram of the FELIG firmware is attached. Its main components are the link (GBT or lpGBT) wrapper, the emulator, the Wupper PCIe engines, and the clock management module. Both the link wrapper and the Wupper, including the register map, are mostly kept the same as in FELIX. The FELIX central router firmware block, responsible for the interface between the link wrapper and the PCIe wupper, was removed and replaced with the emulator. The clock management also differs from FELIX.
The emulator is configurable, allowing to enable and disable channels as well as choose the e-group widths: 2-bit, 4-bit, 8-bit, 16-bit and soon 32-bit; the data format: 8b10b encoding and direct mode; if LSB or MSB is first; the data chunk length; and the generation trigger: internal (using a counter) or external (sent via TTC by the FELIX board connected to FELIG). The data itself is a countdown from the chunk length to zero with an eight bytes header. Some detector specific data generation modules are now being developed to be part of FELIG.
1
Most of the software tools used for FELIX are also available for FELIX since they share the same Wupper and register map blocks. Namely, the initialization procedure for FELIX and FELIG cards uses the same software tool. A toolset called felig-tools was put together to simplify the FELIG configuration procedure. Thus, it is to configure all FELIG relevant registers with simple commands. felig-tools is distributed with the standard FELIX software tools.
For example, a FELIG testing server can host two FLX-712 cards inter-connected via optical links. One of the cards should be running the FELIX firmware version to be tested and connected to a TTC system, while the other is running FELIG.
The GBT version of FELIG, used for the phase-1 FELIX validation, is thoroughly tested and working. Phase-2 FELIG is now being developed. Besides adding the new lpGBT wrapper, it also includes the new FELIX phase-2 wupper and new data formats.
FELIG was already crucial in uncovering some bugs of phase-1 FELIX and is expected to keep being used as a testing and validation tool for the developments coming for phase-2 FELIX.
