Sep 26 – 30, 2011
Vienna, Austria
Europe/Zurich timezone

Development of an ATCA IPMI Controller Mezzanine Board and its usage on an ATCA ROD Evaluator board for the ATLAS LAr upgrade

Sep 27, 2011, 12:15 PM
Room EI 8 (Vienna, Austria)

Room EI 8

Vienna, Austria

<font face="Verdana" size="2"><b>Vienna University of Technology</b> Department of Electrical Engineering Gusshausstraße 27-29 1040 Vienna, Austria


Nicolas Dumont Dayot (LAPP-Laboratoire d'Annecy-le-Vieux de Physique des Particules ()


In the context of the LHC upgrades, a new Read-Out Driver (ROD) board for the ATLAS LAr calorimeter is being developed. xTCA (Advanced/Micro Telecom Computing Architecture) is becoming a standard in high energy physics and is a serious candidate for future readout systems. We will present our current developments to master ATCA and to integrate a large number of very high speed links (96 links/8.5 Gbps) on a ROD Evaluator ATCA board. To manage our ROD Evaluator, we have developed a versatile ATCA IPMI controller for ATCA boards which is FPGA Mezzanine Card (FMC) compliant.

Summary 500 words

The current Read-Out Driver (ROD) system of the ATLAS Liquid Argon calorimeter receives data from 1524 optical links working at a speed of 1.6 Gbps: one link per Front End Board (FEB) and eight links per ROD. These data, from the 182468 calorimeter cells, are processed by DSPs and the results, energy for all cells, time, quality factor and samples for some cells, are transmitted to the readout system over 800 optical links. Data are only received by the ROD on a positive trigger decision (75 kHz mean rate).

In the current model developed to cope with the High Luminosity LHC (HL-LHC) the RODs would receive data for every bunch crossing (40 MHz). Each link from a FEB has then to run at ~100 Gbps and each cell (128 per FEB) has to be treated at a rate of 40 MHz.
Data for the trigger decision would be prepared in the ROD and sent on fast optical links with an adequate structure for trigger calculations. On a positive trigger decision, the results from the RODs would be transmitted to the readout system (~200 kHz mean rate).

We are developing a system in the ATCA (Advanced Telecom Computing Architecture) standard in order to understand its advantages and limitations.
A ROD Evaluator is being designed to exercise the high density of very fast links on a single ATCA board. It requires an Intelligent Platform management Interface (IPMI) controller to respect the ATCA standard. This led us to develop a versatile ATCA IPMI controller for ATCA boards which is FPGA Mezzanine Card (FMC) compliant.
The ROD Evaluator is a full ATCA compliant board with its controlled power supplies, hot swapping and communications with the shelf manager through the ATCA IPMI controller.
The ROD functionality is implemented on a set of blocks based on ALTERA STRATIX-IV-GX FPGAs. A block consists of one FPGA with its 32 bidirectional 8.5 Gbps SERDES and 832 DSP cells, a CPLD to manage FPGA flash configuration and DDR3 memory for data storage.
Two blocks are used with 24 transmitters and 24 receivers connected to AVAGO snap12 10Gbps optical transmitters and receivers. They are linked to the ATCA fabric and update channel by the third block and its SERDES.
This board which is ready to be manufactured will allow us to test fast links and communications on the ATCA backplane. The 832 DSP cells will allow us to implement and test physics algorithms with parameters stored in the DDR3.

The ATCA IPMI controller is built on an FMC card. It is based on a TEXAS microcontroller with an ARM architecture. Its I2C port is used to implement the IPMI functionality.
USB, Boundary Scan, Ethernet are available from the front panel and the latter is also possible from the ATCA base channel. A XILINX SPARTAN-VI FPGA interfaces the microcontroller and the FMC connector to provide versatility for the control of the motherboard (ATCA, configuration, monitoring, etc.). This controller is operational with a set of libraries and is available for other designs.

Primary author

Dr Luis Hervas (CERN)

Presentation materials