The upcoming upgrade of the readout system of the ATLAS experiment at the LHC at CERN is based on the Front-End LInk eXchange (FELIX) system. As part of this upgrade, approximately 120 custom PCIe cards are being produced by an industrial partner, based on a hardware design developed within the collaboration. Such a large production requires detailed Quality Assurance/Quality Control procedures (QA/QC) to ensure the hardware being produced is fully functional and robust.
The Front-End LInk eXchange (FELIX), will form a cornerstone of the upgraded readout system of the ATLAS experiment for the newly installed trigger and detector components after the 2018-2021 shutdown. These systems include the new Muon Small Wheel, Liquid Argon Calorimeter digital readout plus Calorimeter and Muon trigger electronics. The FELIX system is composed of approximately 120 custom PCIe cards hosted by a cluster of commodity servers, with point-to-point connections to detector front-ends at the input stage and an output to a high bandwidth ethernet network. The PCIe card itself hosts a Xilinx Kintex Ultrascale FPGA, up to 8 MiniPOD optical transceivers and a high performance 16 lane PCIe Gen 3 interface. The production of the 120 cards is being undertaken by an industrial partner based on designs developed within the ATLAS collaboration. Each card must undergo a rigorous set of QA/QC tests at the production side, and at CERN, before being accepted for operational use. In order to maximuse production yield and minimise hardware faults, the process has been split into two parts. A 20-board 'pre-series' has been produced, and subjected to the full suite of production tests. This step gives the opportunity to refine the production process and iron out any issues, before initiating the final production run for the remaining 100 cards. In this presentation we will detail the development of the testing platform used to validate the pre-series (which will also be used for the final production).
In more detail, the specific tests used to validate the quality and performance of the cards include:
- verification of the operating temperature of the FPGA,
- measurement of power supply and distribution across the card,
- confirm PCIe bus performance and throughput
- confirm ability to program all on-board devices,
- verify quality of the optical connections via Bit Error Rate tests and Eye Diagram inspection,
- synchronization tests via loop-back configuration using all the optical channels (48 fibers) at high bandwidth.
These measures and more have been specified and a full validation suite prepared, making use of scripting and automation to optimise test time and accuracy.
This full suite has been made available to the contractor in a dedicated test server, with the same setup replicated at CERN. By using these two test benches in tandem it will be possible to rapidly and accurately validate and commission the FELIX cards for use in ATLAS operations.