Speakers
Description
The APOLLO ATCA platform is an open-source design that separates into a generic "Service Module" (SM) and customizable "Command Module" (CM), allowing cost-effective use in applications such as readout of the inner tracker and Level-1 track trigger for the CMS Phase-II upgrade at the HL-LHC. The SM incorporates an intelligent IPMC, robust power entry and conditioning systems, a powerful system-on-module computer, and flexible clock and communications infrastructure. The CM is designed around 2 Xilinx Ultrascale+ FPGAs and high-density, high-bandwidth optical transceivers capable of 25 Gb/s. Crates of APOLLO blades are being tested at Boston University, Cornell University, and CERN.
Summary (500 words)
The APOLLO platform represents a significant advancement in the development of ATCA blades, catering to the rigorous demands of high-energy physics experiments, particularly within the context of the High-Luminosity LHC. The platform is divided into a "Service Module" (SM) and an application-specific "Command Module" (CM), each designed to streamline the implementation and management of complex data acquisition and processing tasks. It addresses key challenges in high-performance ATCA blade development by providing robust power entry and conditioning modules capable of delivering 12VDC at up to 30A, alongside comprehensive cooling solutions to manage the resultant heat effectively. Moreover, the platform supports high-speed data acquisition and communication through optical fiber interfaces, with the CMS-specific design capable of supporting over 100 optical links at speeds up to 25 Gb/s, tailored for tasks that require substantial data throughput and rapid processing capabilities.
The Service Module is a standard-sized ATCA blade that supports a Command Module board within a 7U x 180 mm cutout, incorporating an Enclustra Zynq Ultrascale+ System-on-Module (SoM) with an embedded Linux OS, alongside industry-standard IPMC options such as OpenIPMC, and an Ethernet switch module. The SM’s flexible interface options include front-panel gigabit Ethernet, additional Ethernet to the switch, and four 10 Gb/s bidirectional serial links to the CM. Diagnostic and programming interfaces, such as asynchronous serial, Inter-Integrated Circuit (I2C), and JTAG master/target capabilities, are integrated for routine monitoring and recovery of modules.
The CMS Command Module supports two Xilinx Ultrascale+ VU13P FPGAs in the A2577 package. It has a flexible clocking infrastructure and four 12-wide and one four-wide Samtec Firefly bidirectional sites per FPGA and 54 inter-FPGA links, all rated at 25 Gb/s. Monitoring and low-level control are provided by a Cortex-M3F microcontroller (MCU) running a custom FreeRTOS-based application that independently ensures the safety of the CM. I2C and Universal Asynchronous Receiver-Transmitter (UART) connections between the SOM and MCU afford further flexibility in addition to the 10 Gb/s bidirectional links running the Xilinx Chip-to-Chip protocol. Cooling performance is provided by custom heat sinks. The FPGAs can be programmed via a front panel JTAG connector or directly by the Zynq SOM using Xilinx Virtual Cable protocol.
For software and firmware development, APOLLO provides a comprehensive toolkit that enables new users to quickly become productive. This includes a firmware and software reference design, a set of Advanced eXtensible Interface (AXI) peripherals, and a complete Makefile and script-based build environment. This setup supports easy customization of the Zynq system using a purely text-based system description, which integrates well with repositories and version control systems.
The APOLLO platform serves critical roles in the CMS experiment at the LHC. It will support the Level-1 Track Finding and Inner Tracker Data Trigger and Control systems. Each application requires extensive FPGA resources and is poised to handle substantial power consumption and numerous optical links.
This talk will present the APOLLO production-ready hardware, focusing on test results and a detailed description of the implementations planned for CMS.
