63U ATCA rack thermal performance and its integration in the undergound areas

5 Sept 2019, 14:25
25m
Aula magna

Aula magna

Oral Power, Grounding and Shielding Power, Grounding and Shielding

Speaker

Michal Tomasz Kalinowski ( Tadeusz Kosciuszko Cracow University of technology (PL))

Description

As for the High Luminosity LHC the instantaneous luminosity will be increased by a factor 3, an efficient trigger selection will be crucial. To reduce signal latency, the proximity of the readout electronics to the detector becomes very important. A custom designed 63U rack has been chosen as a house for the ATCA based systems allowing installing 50% more shelves in the detector proximity. The higher power dissipation requires an assessment of the impact on the existing underground infrastructures and cooling capabilities verification in a test setup, which also became the cooling facility to qualify the detector boards prototypes.

Summary

In November 2014, the ATLAS technical coordination launched a project dedicated to the study of the ATCA (Advanced Telecommunications Computing Architecture) integration in the existing LHC rack infrastructure. At the beginning, the project focused on the standard 52U 19” rack equipped with two ATCA shelves cooling capabilities assessment. But over the years, the goals of the test campaign were changing to adjust to the current needs of the detectors groups and to face encountered problems and restrictions such as high noise generated by the shelves and limited cooling water flow in the counting rooms. At a certain stage of the project, it was clear that there will be a need for additional space for shelves in the detector proximity – due to the nature of the counting rooms the only possibility was to go to taller racks.
Therefore, a 63U - 19” rack was designed, ordered and installed in a test facility together with three newly designed adjusted for ATCA dimensions – 2U air/water heat exchangers and three ATCA shelves. Over the months, there were many requests for different tests layouts of which most were addressed. Among them are tests with different computing architectures mixed – ATCA and VME (Virtual Machine Environment), to evaluate if these two very different (in terms of power dissipation and cooling needs) architectures can be combined and work in one rack. Or tests to evaluate the RTM power influence on the overall cooling of the board.
The 63U ATCA rack due to its large power dissipation has a strong impact on the counting rooms existing infrastructures such as HVAC (air conditioning) and water cooling systems, not to mention the noise contamination of the environment – this is why significant part of the test campaign was dedicated to address the unwanted heat release to the counting rooms, and consequently increasing the overall rack cooling efficiency. For this reason different ATCA shelf types were tested to evaluate their cooling capabilities, also tests with air leak tightened rack were carried out to diminish the escaping heat. In order to mitigate the generated noise the soundproofing on the rack level is necessary, the project to further reduce the environment noise contamination is ongoing and its introduction is schedule for the beginning of LS3.
One of the most important goals of this test campaign was to set the limit for the maximum power in front of the boards and on RTM that can be applied per slot to be able to stay within the 50°C temperature target – the results of a measurement campaign with different types of load boards will be presented.

Primary author

Michal Tomasz Kalinowski ( Tadeusz Kosciuszko Cracow University of technology (PL))

Co-authors

Claudio Bortolin (CERN) Jakub Durda ( Tadeusz Kosciuszko Cracow University of technology (PL))

Presentation materials