Speaker
Description
The ATLAS experiment is preparing for the High-Luminosity LHC era, by replacing the current innermost detector with an advanced all-silicon tracker (pixels and strips) to withstand radiation damage and increased particle activity. Pixel module quality control spans various production stages which necessitates a robust data acquisition software capable of handling high data rates and MHz calibrations. Yet Another Rapid Readout (YARR) software, adaptable to diverse hardware platforms including ATLAS Phase-2 readout board i.e. FELIX, facilitates these testing scenarios. This contribution highlights YARR's development, key features, and benchmark performance in calibrating multiple pixel modules using the FELIX readout board.
Summary (500 words)
Yet Another Rapid Readout (YARR) system serves as a Data Acquisition (DAQ) tool, employing a software-centric design and PCIe FPGA boards. Initially developed for the ATLAS detector's Insertable B-layer upgrade in 2014, it facilitated testing and data acquisition at a readout bandwidth of 160 Mb/s. Over time, YARR has evolved into a versatile solution, adaptable to various silicon front-end (FE) types and hardware platforms via PCIe-linked FPGAs. This adaptability enables not only laboratory-scale testing but also full-scale integration on support structures for data transmission tests, ensuring continuity in operations and software maintenance.
Currently, YARR is the designated software for system tests of the ATLAS Phase-2 Inner Tracker (ITk) pre-production and production modules using the official readout electronic board, FELIX. In the architecture of the ATLAS Phase-2 Trigger-DAQ system, FELIX boards will distribute readout data to client software like YARR which will lead to event processing, filtering, and storage. Additionally, YARR will also be able to interact with on-detector electronics, configuring front-ends, performing calibrations at MHz frequencies, and executing diagnostics. FELIX servers will utilize RDMA-based network communication via the netio-next library, following a publish/subscribe model for client applications on the network. Thus, YARR is being developed to communicate with FELIX servers through the felix-client interface for both pixel and strips ITk subdetectors.
YARR-based systems offer deterministic data arrival, suitable for electrical testing of pixel FE modules. Ongoing developments with felix-client interface for network communication and FELIX hardware aim to scale up the number of modules while maintaining the same benchmark timing and performance. Challenges include navigating the network traffic and data buffering delays during calibrations, which necessitate synchronized data packet timing. Aggregating command packets reduces protocol overhead but may increase transmission latency. However, trigger commands are sent synchronously with the calibration injections or consequently with the bunch crossings, for proper readout. As for the readout data, receiving larger buffers increases the data bandwidth. To ensure loss-less readout, YARR has developed a trigger-tagging feedback mechanism, capable of counting events until all trigger tags are received or a timeout occurs.
At LBNL, a dedicated test stand has been established for the development of YARR DAQ software for ITk pixel modules. This setup includes a FELIX board installed on a dedicated server, a Mellanox network interface card enabling RDMA-based or Ethernet-based communication, and an optoboard. It facilitates the testing of data transmission speed and signal quality in a realistic detector configuration. We have achieved successful runs of single and multiple modules in a direct powering mode, completing digital and analog calibrations in under 5 seconds for a nominal trigger frequency of 10 kHz with 100 injections. Our next objective is to scale up the system to run a serially-powered chain with 5 triplet modules (totaling 15 FE chips), utilizing all 4 data lanes for maximum bandwidth at a 5 Gb/s data rate, and another serial power chain of 8 quadruplet modules (totaling 32 FE chips) with various link sharing configurations.
