Speaker
Description
The Pulsar IIb is a custom ATCA full mesh enabled FPGA-based processor board which has been designed with the goal of creating a scalable architecture abundant in flexible, non-blocking, high bandwidth interconnections. The design has been motivated by silicon-based tracking trigger needs. In this talk we describe the Pulsar II hardware and its performance, such as the performance test results with full mesh backplanes from different vendors, how the full-mesh is used for data transfer, how the inter-shelf and intra-shelf synchronization works and the experience gained throughout this process.
Summary
On the Pulsar IIb board a large Virtex-7 (690T) FPGA is directly connected to the RTM, four FMC mezzanine card slots as well as the ATCA full-mesh backplane. The direct connection to the full-mesh enables the full diagnostic capabilities of the MGT transceivers to evaluate the quality of the links at speeds up to 10 Gbps. The abundance of high speed backplane links makes the Pulsar IIb an ideal test platform for characterizing ATCA backplane performance, and to date several vendors have submitted their latest high performance 14 slot full mesh ATCA backplanes to our group for testing. Our backplane test results indicate that channel performance consistency is one of the most significant challenges facing ATCA backplane manufacturers today. The COMTEL “Air/Plane” 100G full mesh backplane, which uses an advanced low loss substrate material, has to date yielded the best and most consistent link performance with all Pulsar IIb boards running all backplane links simultaneously at 10 Gbps without error.
Our Level-1 silicon tracking trigger demonstration system utilizes advanced time multiplexed data transfer schemes to deliver data for a given trigger tower into a single location for processing. The ATCA full mesh backplane offers a unique blend of high performance coupled with a high degree of flexibility. In our current demonstration system we utilize ten Pulsar IIb boards which all receive data from upstream and exchange data over the full mesh backplane. A rear transition module (RTM) which has been designed to support up to 400 Gbps bidirectional communication over fiber optic transceivers which are directly connected to the FPGA. The FMC mezzanine card slots are used for user-defined mezzanine cards, either for more system IO or additional data processing or both. There are a few mezzanine cards developed for the Pulsar II, by us and also by other groups. For example, we have developed one Pattern Recognition mezzanine card, which is a test platform for silicon tracking trigger R&D and supports both ASIC and FPGA (Xilinx UltraScale) based pattern recognition associative memory processors.
System integration testing involves not only sending data between Pulsar IIb boards, but also incorporating the Inter-shelf and Intra-shelf synchronization. In this talk we will present the Pulsar II hardware and its performance, our two full-crate integration tests including the Inter-shelf and Intra-shelf synchronization using CMS trigger framework control hardware, the results of our 40G and 100G ATCA full mesh backplane performance tests, and how the backplane is used for the development of low-latency time-multiplexed data transfer schemes. Today all of the Pulsar IIb serial links operate reliably at 10 Gbps. We must nevertheless continue to refine our layout techniques as FPGA transceivers and ATCA backplanes continue their evolution towards ever higher serial bit rates. We will discuss how the push towards higher serial bit rates has challenged us to focus on signal integrity at the PCB level, and the experiences gained throughout the process.
