Speaker
Description
The CMS detector will undergo an upgrade for Phase-2 of the LHC
program. The back-end electronics will be implemented as ATCA node
boards, connecting to the central systems via a custom `DTH' hub
board.
Instead of a traditional distribution tree, the CMS Phase-2 Trigger
and Timing Distribution System (TCDS) will use a configurable
switching network to connect multiple hardware `run controllers' to
the back-end electronics. This approach increases flexibility by
removing the need for recabling to enable different detector
groupings.
This paper describes the architecture of the TCDS, as well as the
progress made prototyping the DTH and DAQ-800 hardware.
Summary (500 words)
The CMS detector will undergo a major upgrade for the Phase-2 of the
LHC program: the High-Luminosity LHC. The upgraded CMS detector will
be read out at an unprecedented data rate exceeding 50 Tb/s and an
event rate of up to 750 kHz, selected by the Level-1 hardware trigger,
and an average event size reaching 8.5 MB. The Phase-2 CMS back-end
electronics will be based on the ATCA standard, with node boards
receiving the detector data from the front-ends via custom,
radiation-tolerant, optical links.
Traditionally, the CMS trigger and timing system has been implemented
as a distribution tree. For global runs, Level-1 trigger accepts and
synchronisation commands are broadcast by the `run controller'
hardware at the top of the tree to all participating subsystems. Local
(e.g., calibration) runs are made possible by broadcasting alternative
trigger and timing signals down the tree in a particular branch. In
this approach, the cabling configuration defines the possible detector
groupings that can be operated together for such local runs.
The design of the Phase-2 CMS Trigger and Timing Control and
Distribution System (TCDS) replaces the traditional distribution tree
with a configurable switching network. In this design, any of the
(O(16)) run controllers in the system can be combined with any subset
of (O(150)) sub-detector end-points (i.e., groups of back-end
electronics boards). This approach provides increased flexibility,
notably during commissioning, calibration, and
troubleshooting. Detector groupings are created by software selection
instead of cabling changes.
Since the `upstream' direction of the trigger and timing distribution
network is used for the gathering of back-end status information, the
above switching network cannot be implemented as a purely passive
switch. In the CMS case, an FPGA-based approach is chosen: switching
the downstream timing information, while selectively merging the
upstream status information.
One of the challenges of the above approach is the required amount of
FPGA resources for the implementation of the full switch matrix. The
CMS design addresses this by distributing the switching across
multiple FPGAs, on multiple boards.
In order to reduce development effort, prototyping rounds, and the
need for spares, the Phase-2 TCDS will be implemented using hardware
already under development for other tasks in the DAQ system: the DAQ
and Timing Hub (DTH) ATCA hub board, providing an interface to the
trigger, timing, and DAQ systems, and a DAQ-800 node board providing
additional DAQ throughput. For this purpose, auxiliary functionality
was added to these boards, taking care not to affect their original
functionality. The DTH, designed to interface back-end electronics to
the central trigger, timing, and DAQ systems, will be reused to
interface the TCDS controller and switch boards to the Level-1
trigger, to the luminosity monitoring systems, and to the HL-LHC RF
system. The DAQ-800 DAQ board will be used as powerful FPGA platform
for the implementation of the controllers and the switching
network. The Samtec Firefly connections intended to receive event data
from sub-detector back-ends will be equipped with QSFP+ adapters
interfacing to (SFP+ optics in) the DTHs in all sub-detector back-end
crates.
