Balazs Voneki (CERN) Sebastien Valat (CERN)
The LHCb experiment is preparing a major upgrade in 2020 resulting in a need for a high-end network for a data acquisition system. Its capacity will grow up to a target speed of 40 Tb/s, aggregated by 500 nodes. This can only be achieved reasonably by using links capable of coping with 100 Gigabit/s line rates. The constantly increasing need for more and more bandwidth has initiated the development of several 100 Gigabit/s networks mostly for the HPC field. There are 3 candidates on the horizon, which need to be considered: Intel® Omni-Path, 100G Ethernet and EDR InfiniBand. We present test results with such links both using standard benchmarks (e.g. iperf) and using a custom built benchmark called LHCB-DAQPIPE. DAQPIPE allows to emulate various classical event-building protocols, push, pull, barrel-shifter etc… on multiple technologies. It is particularly well suited to run on supercomputing sites, which are the only possibility to test systems which have the same size as the required DAQ networks. Such systems can simply not be afforded for lab-tests. The key benefit of these measurements is that we can gain detailed insight into the behaviour of the system without the need to build a system to scale. This allows to find out the limitations of the different network component and how they are connected with protocols.