Sep 23 – 27, 2013
Perugia, IT
Europe/Zurich timezone

Evaluation of 400m, 5Gbit Versatile Link lengths over OM3 and OM4 fibres for the LHCb upgrade

Sep 25, 2013, 5:10 PM
1m
Perugia, IT

Perugia, IT

<font face="Verdana" size="2.5">Congress center Giò Via R. D'Andreotto, 19 06124 Perugia (PG) Italy
Poster Poster

Speaker

Rainer Schwemmer (CERN)

Description

The LHCb experiment will upgrade its DAQ system to a trigger less, 40MHz read-out after LS2. To be able to process the approximately 40Tbit/s of data we will require a massive computing farm. This computing farm can not be installed underground, in the vicinity of the detector anymore due to the enormous power and cooling requirements. An affordable data transport solution has to be found to carry the data from the detector to the new data center on the surface. The distance to cover is estimated to be between 300 and 400 meters. We evaluated the feasibility of using the 5Gbit/s Versatile Link to cover the full distance over OM3 and OM4 quality fibres and will present our results.

Summary

Since LHCb will have to install their new computing center on the surface instead of the underground cavern, next to the detector, we will have to come up with a solution of transporting 40Tbit/s to the surface based data center, which is approximately 400m away from the detector. We have looked at several commercial, high speed interconnects, but all current solutions are prohibitively expensive at these distances. Further, all future solutions like 100 Gbit Ethernet or Infiniband will reach even shorter distances and would require expensive LR optics components.

The Versatile Link offers a way out of this dilemma. It was originally developed as a low mass, low power link that can handle the radiation levels and data rates of LHC's particle detectors. To be able to cope with radiation, the link runs at a relatively low speed of 5 Gbit/s. Since the extinction ratio and optical lauch power of the Versatile Link are comparable to other commercial SR based optics, this should at the same time allow longer distances than commercial links.

There are of course several questions that need to be answered before this system can be deployed on a large scale. The link length is at the limit of the specification for 10Gbit over OM4. The closest commercial link would be 4Gbit fibre channel, which is also rated at 400m over OM4. Additionally this mode of operation was not the original intent of the Versatile Link, which is supposed to connect the LHC detectors to relatively close by network translators.

We decided to test several different fibres of different manufacturers to see if this approach is possible. We also tested if OM3 is a viable candidate for 5 Gbit/s link speeds. We will present the results of our investigation and show that it is indeed possible to use the Versatile Link in a long distance setup over OM4 as well as OM3.

Primary author

Co-authors

Csaba Soos (CERN) Dr Jan Troska (CERN) Jean-Pierre Cachemiche (Centre National de la Recherche Scientifique (FR)) Ken Wyllie (C) Niko Neufeld (CERN)

Presentation materials