30 September 2024 to 4 October 2024
San Diego Supercomputer Center, US
US/Pacific timezone

Adopting RUCIO for the Italian datalake: National experience and perspectives

30 Sept 2024, 14:20
20m
San Diego Supercomputer Center, US

San Diego Supercomputer Center, US

Speaker

Antonino Troja

Description

With four national laboratories and twenty divisions, the National Institute for Nuclear Physics (INFN) is one of the largest research institutions in Italy. INFN has been running for more than 20 years a distributed computing infrastructure (WLCG Tier-1 at Bologna-CNAF plus 9 Tier-2 centers) which currently offers about 200k CPU cores, 250 PB of enterprise-level disk space and 200 PB of tape storage, serving more than 100 international scientific collaborations. Some of the resources provided by each INFN computing center are integrated in world wide Grid infrastructures, while other resources are exposed through Cloud interfaces.

Thanks also to the "Italian National Centre on HPC, Big Data and Quantum Computing" and the TeRABIT projects (both funded in the context of the Italian "National Recovery and Resilience Plan") INFN is now revising and expanding its infrastructure to tackle the computing challenges of the next decades and to harmonize the management of the grid and cloud stacks. This process involves also the federation of the national computing centers such as CINECA HPC and its pre-exascale Leonardo machine.

In this context RUCIO and its ancillary services have been identified as a key architectural component of the national datalake architecture.

This talk will first present our operational experiences with RUCIO, from deployment up to heterogeneous backend federation, passing through the "onboarding" of the first scientific communities. In addition, we will show how we addressed a few specific requirements such as authorization models, user registration and support for external metadata catalogs. Finally, we will provide an insight on how to extend the adoption of a Rucio-based data management infrastructure at national Level.

Authors

Alessandro Italiano Andrea Rendina Antonino Troja Bernardino Spisso (University Federico II and INFN, Naples (IT)) Daniele Spiga (Universita e INFN, Perugia (IT)) Diego Ciangottini (INFN, Perugia (IT)) Federica Fanzago (Universita e INFN, Padova (IT)) Lucia Morganti Marco Verlato (Universita e INFN, Padova (IT)) Massimo Biasotto (INFN Legnaro) Massimo Sgaravatto (Universita e INFN, Padova (IT)) Michele Delli Veneri Nadir Marcelli (INFN Roma 1) Stefano Stalio

Presentation materials