Speaker
Description
Xilinx Zynq SoCs are used by the CMS TDAQ in its back-end electronics since LHC Run-2, between 2015-2018. For the Phase 2 upgrade of the LHC, about 1000 devices will be deployed, comparable to the number of High Level Trigger (HLT) nodes today. This scale presents challenges for the SoC integration in the experiment network, system administration, network management, booting process and root file system management among others. We present an evaluation of various Linux distributions (PetaLinux, Yocto, ArchLinux, CentOS) and a proposal of how to address the challenges involved in developing and maintaining custom linux distributions for CMS.
Summary
System on Chip (SoC) devices have become popular for custom electronics HEP boards since the advent of FPGAs with integrated CPUs. Advantages include the tight integration between the FPGA and the CPU, high flexibility in terms of hardware design, and the relatively powerful CPUs which are capable of running fully fledged operating systems like CentOS, Archlinux and even Ubuntu.
SoCs provide a powerful hardware fulfilling the needs for configuration, control and monitoring from the perspective of Trigger and Data Acquisition.
The CMS Trigger and Data Acquisition System already features a small number of back-end Xilinx Zynq SoCs, in use since LHC Run-2 in 2015-2018. These are stand-alone and isolated installations. For the High Luminosity phase of the LHC, completely new and upgraded CMS back-end electronics, with auxillary Xilinx Zynq devices, will be built. About 1000 such SoC devices are expected to be deployed. This is comparable to the number of High Level Trigger(HLT) nodes today.
The scale of the project poses challenges for the SoC integration in the experiment network, system administration services, networking configuration and management. Typical challenges like time distribution, IP/Name distribution (DHCP or other), unique identification of the devices, security, remote/authorised access, remote system logs, read-only or read-write root file systems, NFS mounted root or application file systems, local or network system boot, and configuration management form the crux of our research. Furthermore, with the emergence of more powerful embedded CPUs, it will be interesting to assess how much of the data acquisition software stack could, or should, be deployed on those devices compared to server PCs.
A few Linux distributions (compatible with ARM 32 and 64 bit architectures) which could be of interest for SoC devices in the context of CMS, have been indentified. The major distributions include PetaLinux, Yocto, ArchLinux, CentOS.
In this paper we will propose a strategy to address the complexity of building a distribution, discuss the requirements on hardware resources, and aspects related to network and sysadmin integration in the context of CMS.
