Speaker
Description
Summary
High luminosity conditions at the LHC pose many unique challenges for potential silicon based track trigger systems. This is true for both Level-1 and Level-2 trigger applications. Among those challenges is data formatting, where hits and clusters from many thousands of silicon modules must first be shared and organized into overlapping eta-phi trigger towers due to finite size of the beam’s luminous region in z and the finite curvature of charged particles in the magnetic field. Communication between nodes requires high bandwidth, low latency, and flexible real time data sharing.
The first silicon based track trigger at the LHC will be the ATLAS Fast Tracker (FTK) at Level-2. Although FTK is designed for Level-1 Accept rates up to 100kHz, the data volume per event is quite large since all silicon modules (more than 86 million channels) are involved at high luminosity, therefore this is where challenging data formatting issues will be encountered for the first time. We have been developing data formatting solutions for high luminosity LHC conditions and the FTK Data Formatter system is the first targeted application. Early in the design process our simulations showed that sharing between Data Formatter nodes is asymmetric and highly dependent upon upstream cabling and detector geometry. A high bandwidth full mesh backplane is a natural fit for the Data Formatter system.
We have selected the Advanced Telecom Computing Architecture (ATCA) platform as it supports a robust full-mesh backplane in a reliable industry standard form factor. A custom ATCA Data Formatter board, called the Pulsar IIa, is designed around Field Programmable Gate Arrays (FPGAs). These FPGAs feature many high speed serial transceivers which are directly connected to the full mesh backplane and to fiber optic transceivers on a rear transition module (RTM). The overall design goal is to create a uniquely scalable architecture abundant in flexible, non-blocking, high bandwidth board to board communication channels while keeping the design as simple as possible.
Expandability and scalability are achieved through three mechanisms. First, each board supports up to four mezzanine cards connected to the main FPGAs. Each mezzanine card may contain FPGAs, pattern recognition ASICs, fiber optic transceivers, or any other custom hardware. Our mezzanine cards use the FPGA Mezzanine Card (FMC) standard which has become popular with Xilinx development boards and many third party vendors. Secondly, additional boards may be installed in the crate. Unlike a shared bus system, adding boards to the mesh network has a minimal impact on the system latency while dramatically increasing system processing power and I/O capability. Lastly, we have reserved several transceivers on rear transition modules (RTM) for dedicated serial links between boards in different crates.
The performance of the Data Formatter design meets and exceeds the FTK requirements. This high performance scalable architecture may find applications beyond tracking triggers, and may serve as a starting point for future Level-1 silicon based tracking trigger R&D for CMS and ATLAS. In this talk we describe our design methodology, prototype test results and experiences designing our first ATCA board.
