Mar 6 – 8, 2023
Europe/Zurich timezone

C(ERN) BACK(UP): consolidated multi-petabyte backup solution for heterogenous storage and filesystems

Mar 8, 2023, 9:45 AM
15m
Presentation Scalable Storage Backends for Cloud, HPC and Global Science Scalable Storage Backends

Speaker

Gianmaria Del Monte (CERN)

Description

The IT storage group at CERN is resposible to ensure integrity and security of all the stored data for physics and general computing services. In the last years a backup orchestrator, cback, has been developed based on the open source backup software restic. Cback is able to backup EOS, CephFS and any local mountable file system, like NFS or DFS. cback is currently used to daily backup CERNBox data (2.5 billion of files and 18PB), including experiment project spaces and user home directories, CephFS Manila shares, the CVMFS home folders and the CERN gitlab instance.

The data copy is stored in a disk-based S3 cluster in another geographical location in the CERN campus 4km away from the main data center (protecting against natural disasters). The usage of restic allows us to reduce the storage costs thanks to the deduplication of the data. In the last months, the cback portal server has been implemented, exposing a set of REST APIs to allow the intergration with end-user backup utilities to navigate snapshots and restore data.

In this presentation, we will describe the architecture and the implementation of cback, the integration with CERN services and the future integration with tape archive for long term data preservation.

Primary author

Co-authors

Presentation materials