Speakers
Description
For the Cherenkov Telescope Array Observatory (CTAO), Rucio has been selected as the core technology for managing and archiving raw data collected from the La Palma and Chile telescope sites. It is a key component of the Bulk Data Management System (BDMS) within the Data Processing and Preservation System (DPPS), providing efficient storage, transfer, and replication. In addition to raw data and auxiliary data from the Cherenkov cameras, Rucio handles higher-level data and simulated data from DPPS's workload and pipelines, with combined data volumes projected to reach several petabytes per year.
In this talk, we share our experience deploying and operating Rucio for BDMS prototyping in Docker and Kubernetes. We highlight the successful implementation of data transfers and replication to grid-based storage elements (RSEs), utilizing Rucio's conveyor daemons, FTS server, X.509 certificates, and proxies. Through unit and integration tests in a GitLab CI environment, we validated Rucio's functionalities, and we outline future plans for deploying Rucio’s to DPPS data centers to meet the evolving needs of CTAO's BDMS.
