Speaker
Description
In this contribution, we present a new Rucio-based service designed specifically to simplify data management for the Small and Medium experiments at CERN.
Rucio has become the de-facto data management solution for major experiments in high-energy physics and related scientific domains such as astrophysics, providing a scalable, policy-driven framework for distributed data placement, access, and lifecycle management. Small and medium-sized experiments have similar challenges to those of big experiments in data organization, reliability, and long-term preservation. These experiments seek a robust data management platform without the operational overhead associated with large-scale deployments.
Two years ago we had the idea of creating a new class of service for these experiments and we finally can report on this new successful service based on Rucio, EOS and CTA (CERN Tape Archive) and FTS (File Transfer Service).
The service offers a low barrier to adoption: experiments write data into a designated disk pool, and archiving, replication, and policy enforcement are handled automatically. This approach creates the illusion of a hierarchical storage manager, allowing data acquisition to write into a disk pool at which point Rucio takes over, managing scalable disk workflows and flexible tape-backed archival.
The service also acts as a blueprint for new experiments, such as ShiP, providing a ready-to-use, production-grade data management environment from the beginning. At the same time, existing experiments (AMS, NA62) , benefit from a managed service, reduced operational effort, and modernization of their data management workflows.
We describe the system’s architecture, the integration model that minimizes data migration and lessons learned while deploying the services for AMS, ShiP and NA62 experiments.