The new tape archive service, ANTARES (A New Tape ArchivE for STFC), at RAL Tier-1 went into production on 4th March, 2022. The service is provisioned with EOS-CTA, developed at CERN. The EOS cluster, a “thin” SSD buffer, manages incoming namespace requests and CTA provides the tape back-end system responsible for the scheduling and execution of tape archival and retrieval operations. In this...
Increasing user demand for file-based storage, provided by the STFC Cloud at RAL, has motivated the production of a new shared file system service based on OpenStack Manila. The service will be backed by a new all-SSD Ceph cluster, ‘Arided’, deployed using the Cephadm orchestrator. This talk will provide a brief overview of our experience deploying a test instance of this service using a...
The presentation will summarize highlights from the 6th EOS workshop and discuss evolution of EOS services and the development roadmap during Run-3.
EOS services are used by large user communities and in many cases exposed and operated as a very large shared resource - though the criticality of individual IO activities varies. To give operational handles to shape data access by activity we have recently added support for direct IO, IO priorities, bandwidth policies and filesystem stream overload protection. For meta-data access EOS...
Jiangmen Underground Neutrino Observatory (JUNO) is an under-construction neutrino experiment located in Jiangmen, China, which is expected to generate about 3 PB experimental data per year. JUNO plan to share those data to all JUNO collaborators from 4 main data centers in China, France, Italy and Russia.
Distributed data management system with Third-Party-Copy (TPC) data transfer support...
Bulkrequests is a small tool that communicates with dCache through its REST API. It arises from the need to be able to consult and modify in a massive way the qos and locality of files stored on tape, such as to pin or unpin a set of files to/from disk as required. It was designed to cover this need in a simple way through a command line tool waiting for the new dCache bulk REST API that...
LHC Run 3 is imposing unprecedented data rates on the tape infrastructure at CERN T0. Here we report on the nature of the challenge in terms of performance and reliability, on the hardware we have procured, and how it is deployed, configured and managed. We share details of our experience with the technology selected, a mix of IBM and SpectraLogic libraries and Enterprise and LTO drives. In...
During the ongoing long shutdown, all elements in LHC data-taking have been upgraded. As the last step in the T0 data-taking chain, the CERN Tape Archive (CTA) has done its homework and redesigned its full architecture in order to match LHC Run 3 data rates.
This contribution will give an overview of the CTA service and how it has been deployed in production. We discuss the measures taken...
The ever increasing amount of data that is produced by modern scientific facilities like EuXFEL or LHC puts a high pressure on the data management infrastructure at the laboratories. This includes poorly shareable resources of archival storage, typically, tape libraries. To achieve maximal efficiency of the available tape resources a deep integration between hardware and software components...
Physics analysis is done at CERN in several different ways, using both interactive and batch resources and EOS for data storage. In order to understand if and how the CERN computer centre should change the way analysis is supported for Run3, we performed several performance studies on two fronts: measuring the performance and utilisation levels of EOS with respect to the current analysis...
This presentation will provide a short overview and comparison of four available Open Source erasure coding technologies for storage (MINIO, RADOS, EOS, XRootd EC) in the context of the Erasure Coding Working Group.
Over the last years we have observed increasing importance of object storage in the WLCG community. In this contribution we report on our effort to accommodate object storage use cases within XRootD, a software framework that is a critical component for data access and management at WLCG sites. Firstly, we introduce a high performance erasure coding (EC) based file storage module motivated by...
Database systems have been known to deliver impressive performance for large classes of workloads. Nevertheless, database systems with mammoth data sets or high throughput applications can challenge the capacity of a single server. High query rates can exhaust the CPU capacity of the server and having working set sizes larger than the system's RAM stresses the I/O capacity of disk drives. This...
