The ESRF has undertaken the implementation of a data acquisition framework for 2D-Xray detectors called RASHPA. It allows detectors to push data directly into one or more backend computers due to its RDMA feature. In principle, RASHPA is independent of the high speed data link type as long as it supports RDMA and network routability in order to dispatch data to multiple destinations. In a preliminary prototype, PCIe-over-cable was selected to be the candidate.
Despite the benefits of this type of link, for which the native RDMA feature is the most important, it presented strong limitations including the availability of COTS products in particular PCIe-over-cable adapters and switches in addition to the lack of standardization of optical cabling form.
Ethernet protocol is a standard way of communication and widely used in nowadays networks. Thanks to recent FPGA generations that embed a 100G MAC IP, the integration of such high bandwidth data link in next detector generations becomes possible. RDMA over Ethernet is a subject of many recent projects such as RoCE and iWarp.
In this work, the hardware implementation of a basic RDMA over UDP/IP Ethernet transmitter/receiver logic on a KCU116 (kintex ultrascale+) Xilinx development board is presented. A commercial 100Gb Mellanox board (MCX415A-CCAT) is used together with wireshark software to analyze the received packets. Obtained results show that one can reach a maximum stable performance of 95 Gbps with minimum packets size of 32KB. In the final paper, the FPGA implementation of the 100Gb Ethernet network will be detailed.