Speaker
Description
Summary
The ALICE Read-Out Receiver Cards (RORCs) used during Run1 implement the Detector Data Link (DDL) protocol via two optical links as interface to the detector frontend electronics and a PCI-X or PCIe Gen1 interface to the host PC. All previous versions of these boards are limited in link speed around the currently used 2.125 Gbps and the PCI-X interface used by some of the boards is hardly available any more in recent server systems. Coming towards Run2, some detectors are considering a higher read-out link rate. In order to overcome the interface limitations and provide room for link speed upgrades a new board has been developed as a common project of Data Acquisition and High-Level Trigger.
This Common Read-Out Receiver Card (C-RORC) holds a Xilinx Virtex-6 FPGA coming with a PCI-Express 8 lane Gen2 interface to the host PC and serial transceivers able to run at up to 6.6 Gbps. An optical interface with 12 links using parallel optical modules has been implemented to cover a complete TPC sector of 6 links with a single board on DAQ side while sending a copy of all data to the HLT with the remaining 6 links. This improved data locality allows earlier combination of data from different links and simplifies data collection for later processing stages. Detector data is transported from the FPGA into the host memory using Direct Memory Access (DMA). The FPGA provides sufficient logic resources to implement a 12 channel DMA engine and the possibility to do online data preprocessing. A flexible configuration management system ensures the correct configuration of the FPGA and allows to monitor, change and re-trigger the FPGA configuration process through sideband signals even if the PCIe interface is down. Two independent on-board DDR3 sockets for regular SO-DIMM memories as used in most mobile computers provide additional fast storage to the board.
The first C-RORC prototypes have been produced and could already be tested extensively. The board is detected reliably as 8 lane Gen2 PCIe device from the host machine and the custom DMA engine allows to push data into the host memory using the full available PCIe bandwidth. The DMA engine firmware implements scatter-gather DMA to allow non-continuous physical memory on the host machine. The software interface is realized with a custom microdriver and a userspace library. The compatibility and the throughput of the optical links using the DDL protocol could be verified with a test setup at CERN where the C-RORC was connected to the existing readout hardware. Higher optical link rates could be tested with pseudo-random bit sequences using two interconnected C-RORC boards. The DDR3 memory sockets could confirmed to be working with different SO-DIMM modules up to 1066 Mbps.
Summarizing all hardware tests the board is perfectly suitable for operation in the current ALICE readout architecture and enables upgrades of detector readout links and preprocessing algorithms. Furthermore, the ATLAS experiment has comparable hardware requirements on the upgrade of their readout system and is strongly considering to also use this newly developed C-RORC.
This Common Read-Out Receiver Card (C-RORC) holds a Xilinx Virtex-6 FPGA coming with a PCI-Express 8 lane Gen2 interface to the host PC and serial transceivers able to run at up to 6.6 Gbps. An optical interface with 12 links using parallel optical modules has been implemented to cover a complete TPC sector of 6 links with a single board on DAQ side while sending a copy of all data to the HLT with the remaining 6 links. This improved data locality allows earlier combination of data from different links and simplifies data collection for later processing stages. Detector data is transported from the FPGA into the host memory using Direct Memory Access (DMA). The FPGA provides sufficient logic resources to implement a 12 channel DMA engine and the possibility to do online data preprocessing. A flexible configuration management system ensures the correct configuration of the FPGA and allows to monitor, change and re-trigger the FPGA configuration process through sideband signals even if the PCIe interface is down. Two independent on-board DDR3 sockets for regular SO-DIMM memories as used in most mobile computers provide additional fast storage to the board.
First C-RORC prototypes have been produced and could already be tested extensively. The board is detected reliably as 8 lane Gen2 PCIe device from the host machine and the custom DMA engine allows to push data into the host memory using the full available PCIe bandwidth. The DMA engine firmware implements scatter-gather DMA to allow non-continuous physical memory on the host machine. The software interface is realized with a custom microdriver and a userspace library. The compatibility and the throughput of the optical links using the DDL protocol could be verified with a test setup at CERN where the C-RORC was connected to the existing readout hardware. Higher optical link rates could be tested with pseudo-random bit sequences using two interconnected C-RORC boards. The DDR3 memory sockets could confirmed to be working with different SO-DIMM modules up to 1066 Mbps.
Summarizing all hardware tests the board is perfectly suitable for operation in the current ALICE readout architecture and enables upgrades of detector readout links and preprocessing algorithms. Furthermore, the ATLAS experiment has comparable hardware requirements on the upgrade of their readout system and is strongly considering to also use the C-RORC.
