The original CMS Timing, Trigger, and Control (TTC) system provided data-taking synchronisation during Run1 of the LHC. Detector upgrades foreseen to be completed in the coming years require a greater number of detector partitions than were available in the TTC system. This along with the upcoming upgrade of the CMS Global Trigger (GT) allowed the re-definition of the boundary between the TTC and GT systems and separate the physics trigger functionality from that of controlling and synchronising the data-taking of CMS. The new CMS Timing and Control Distribution System (TCDS) has been designed, built and installed in CMS during LS1 and is in routine operation.
The TCDS distributes timing and control data flowing to the detector front-ends and receives back status information related to the data-taking readiness of the detector systems. The clock reference that is synchronous to the LHC is distributed along with the fast control information required to keep the data-taking in step across the CMS detector partitions. The detector systems provide basic status information (TTS) regarding their readiness for accepting more triggers by sending data back to the TCDS in the upstream direction.
Physics triggers are received from the GT system by the Central Partition Manager (CPM) module that is the controller responsible for orchestrating global data-taking in CMS that involves multiple sub-detectors. The system clock (received from the LHC RF systems) is also received by the CPM for further distribution. Residing in the same crate are a number of Local Partition Managers (LPMs) that are able to orchestrate local data-taking for a particular sub-detector in CMS. Each LPM contains eight identical firmware blocks that translate the generic synchronisation commands emitted by the Partition Manager (Central or Local) into the sub-detector specific commands that are understood by the detector front-ends. The final module in the chain (of which there is one per detector partition) is the Partition Interface (PI). The PI serves as a fan-out for downstream TTC data and makes an intelligent OR of the status information (TTS) coming back.
The TCDS is based on the uTCA family of standards. In the partition manager crate, the backplane is used to distribute clock and fast timing commands from the CPM to the LPMs. The backplane is also used to transmit 5Gb/s serial data used to synchronise the new CMS LumiDAQ system and to send TTS information from the LPMs to the CPM.
The TCDS was installed in CMS in two phases: in the first a demonstrator was installed in parallel with the existing TTC system to allow sub-detectors to become familiar with the TCDS hardware and software functionality; and in the second the TTC system was removed and replaced with the full TCDS. Details of the rack, powering and monitoring provided for the TCDS system will be described. This was the first time a major system in CMS installed uTCA infrastructure and thus provided a model to be copied in subsequent installations.