Speaker
Description
ALICE is preparing a major upgrade for 2021.
Subdetectors upgrading their counting room DAQ electronics will use a
common hardware to receive physics data: the Common Readout Unit (CRU).
The same CRU will also distribute the LHC clock and trigger to many of
the upgrading subdetectors (~7800 front end cards).
Requirements are strict: for the clock the allowed jitter (RMS) is
typically <300ps, and <30ps for timing critical subdetectors; the
allowed skew is typically <1ns, and <100ps for timing critical
subdetectors. A constant latency for distributing the trigger is a
must.
Techniques used to meet these requirements will be presented.
Summary
In the upgraded system the CRUs will sit in the data acquisition computers in the counting room.
In the upstream direction (front end to DAQ) they will receive physics measurement data on up to 36 optical links (mostly using GBT protocol, but in some cases using detector specific custom protocols), then pre-process, and finally forward data to the host PC's memory.
In the downstream direction (DAQ to front end) the CRUs will receive from the trigger system the LHC clock, trigger decisions, and flow control messages over a custom Passive Optical Network (PON).
The LHC Clock will be recovered, and used as a reference clock for the downstream GBT links towards front ends, thus implementing the clock forwarding.
In typical FPGA applications the high speed transceivers are used for source synchronous data transmission: clock accompanying the data is combined with and embedded into the serial data stream.
As a characteristic of these source synchronous designs, users typically do not care about the phase relation of the reference clock and the transceivers' RX/TX clocks, or the latency for crossing between these clock domains, or about the minute variation of all these.
In our application these variations are critical, hence we are using FPGAs in an unusual way, for which these devices are not optimized. All Process, Voltage, and Temperature (PVT) variations must be controlled, to be able to meet our strict system requirements.
This is not only true for the FPGA and the design running in it, but also for all external components making up the full clock and trigger distribution chain: optical transceiver modules, external fanout buffers, multiplexers, and jitter cleaners for the clock.
This requires special attention to component selection, clocking architecture, PCB design, and clock domain crossing techniques inside the FPGA.
These considerations, the actual design decisions made, and
characterization measurement results will be presented.
