Speaker
Niko Neufeld
(CERN)
Description
The LHCb Data Acquisition (DAQ) will be upgraded in
2020 to a trigger-free readout. In order to achieve this goal we will need to connect 500 nodes with a total network capacity of 40 Tb/s. To get such an high network capacity we are testing zero-copy technology in order to maximise the theoretical link throughput without adding excessive CPU and memory bandwidth overhead, leaving free resources for data processing. More CPU power available means less machines needed for accomplishing the same task resulting in less power, space and money used for the same result.
We had developed two test applications one using non zero-copy protocols (TCP/UDP) and the other using the OFED libibverbs API, which can provide low level access and high throughput. The libibverbs API offers a good level of flexibility allowing the application to be compatible with different RDMA solutions, like Infiniband and Internet Wide Area RDMA Protocol (iWARP), and it provides us the possibility to perform tests on different technologies using the same application for a more comprehensive evaluation of different implementations of an RDMA protocol over different network technologies.
We will present throughput, CPU and memory overhead measures comparing Infiniband and 40 GbE solutions using RDMA, those measures will be present for several network configurations to test the scalability of the system.
The comparison between zero-copy and non zero-copy
results will be presented to evaluate the impact of
high speed Ethernet communication (40 Gb/s now and 100
Gb/s then) on the host machine in terms of CPU and
memory usage. These results are relevant to wide range
of high-speed, low-cost PC based data-acquisition.
Primary author
Flavio Pisani
(Universita e INFN, Roma I (IT))
Co-authors
Daniel Hugo Campora Perez
(CERN)
Niko Neufeld
(CERN)
