11–13 Mar 2024
CERN
Europe/Zurich timezone

The S3 Object Storage Service on INFN Cloud

11 Mar 2024, 16:15
15m
503/1-001 - Council Chamber (CERN)

503/1-001 - Council Chamber

CERN

162
Show room on map
Presentation CS3 Community Site Reports Services and Infrastructures

Speaker

Dr Alessandro Costantini (INFN-CNAF)

Description

Backed by the 20 years of successful development and operation of the largest Italian research e-infrastructure through the Grid, the Italian National Institute for Nuclear Physics (INFN) has been running for the past three years INFN Cloud, a production-level, integrated and comprehensive cloud-based set of solutions, delivered through distributed and federated infrastructures. 

INFN Cloud provides a large and customizable set of services, ranging from simple IaaS to specialized SaaS solutions, centered through a PaaS layer built upon flexible authentication and authorization services, offered via INDIGO-IAM, and optimized resources and services orchestration.

Since its beginning, INFN Cloud has offered its users and collaborations an S3-based Object Storage service for data archiving.Such S3 buckets can be accessed  via a web ui or programmatically. They can also be mounted as volumes in a semi-posix fashion, providing a sort of “geographic home directory”, which can be accessed remotely e.g.  on Jupyter Notebooks.

Key features of the S3 service are (i) OIDC authentication via the Indigo DataCloud IAM service, (ii) the use of Open Policy Agent for fine grained authorization, (iii) full integration with other INFN Cloud services, and (iv) data replication over two data centers 400km away.

Recently, the service has been migrated from a Minio Gateway on top of a distributed OpenStack Swift cluster to a multisite CEPH RGW infrastructure supported by a web user interface that has been developed in house.

We will describe the main features of our setup, focusing on the authentication/authorization model and the most prominent use cases, comparing the pros and cons of the new solution we adopted in respect to the one we abandoned.

Primary authors

Ahmad Alkhansa (INFN - CNAF) Dr Alessandro Costantini (INFN-CNAF) DIEGO MICHELOTTO (INFN - National Institute for Nuclear Physics) Daniele Spiga (Universita e INFN, Perugia (IT)) Diego Ciangottini (INFN, Perugia (IT)) Federico Fornari Giada Malatesta Jacopo Gasparetto (CNAF) Massimo Sgaravatto (Universita e INFN, Padova (IT)) Stefano Stalio

Presentation materials