Sharing your scientific data with Rucio

28 Jan 2020, 10:40
20m
Presentation User Voice: Novel Applications, Data Science Environments & Open Data Fabric and platforms for Global Science

Speakers

Mario Lassnig (CERN) Martin Barisits (CERN) Thomas Beermann (Bergische Universitaet Wuppertal (DE)) Cedric Serfon (Brookhaven National Laboratory (US))

Description

Rucio is an open-source software framework that provides scientific
collaborations the functionality to organize, manage, monitor, and
access their distributed data across heterogeneous infrastructures.
Rucio was originally developed to meet the requirements of the
high-energy physics experiment ATLAS, and is continuously extended to
serve a diverse set of scientific communities, from agricultural to
radioastronomy. In 2019, Rucio orchestrated more than an Exabyte of data
across a billion files on 130+ data centres.

In this contribution we want to address potential future improvements to
scientific data managed with Rucio: (1) transparent provisioning of data
for interactive analyses, (2) publishing and annotation of data
according to FAIR principles, and (3) selective synchronisation of data
for users and desktop applications. A special focus across all three
topics will be dynamic adaptation of dataflows to protect global system
performance.

Primary authors

Mario Lassnig (CERN) Martin Barisits (CERN) Thomas Beermann (Bergische Universitaet Wuppertal (DE)) Cedric Serfon (Brookhaven National Laboratory (US))

Presentation materials