4th Rucio Community Workshop (Virtual)
Rucio is a software framework that provides functionality to organize, manage, and access large volumes of scientific data using customisable policies. The data can be spread across globally distributed locations and across heterogeneous data centers, uniting different storage and network technologies as a single federated entity. Rucio offers advanced features such as distributed data recovery or adaptive replication, and is highly scalable, modular, and extensible. Rucio has been originally developed to meet the requirements of the high-energy physics experiment ATLAS, and is continuously extended to support LHC experiments and other diverse scientific communities.
We set up a mailing list to which you can subscribe and where we will send more details about the program in the coming weeks.
We also created a Slack channel dedicated to the workshop discussion on the Rucio Slack workspace (Invitation Link). Join #workshop.
-
-
15:00
→
15:45
Opening & Closing
- 15:00
- 15:05
-
15:15
Rucio: State of the Union 30mSpeaker: Martin Barisits (CERN)
-
15:45
→
16:30
Community ReportsConvener: Alastair Dewhurst (Science and Technology Facilities Council STFC (GB))
-
15:45
CMS 15mSpeaker: Eric Vaandering (Fermi National Accelerator Lab. (US))
-
16:00
Rucio for the Light Dark Matter eXperiment (LDMX) 15m
The Light Dark Matter eXperiment (LDMX) is a planned small-scale accelerator-based experiment to search for dark matter in the sub-GeV mass region. Finalizing the design of the detector relies on Monte-Carlo simulation of expected physics processes. A distributed computing pilot project was initiated based around existing software used by other communities, including Rucio for data management. Rucio is primarily used as a dataset catalog, and LDMX also makes extensive use of Rucio metadata to record physics properties as well as bookkeeping information on the data produced. In this talk we describe how Rucio is used by LDMX and propose some possible extensions to the metadata functionality.
Speaker: David Cameron (University of Oslo (NO)) -
16:15
DUNE Rucio Usage and Status 15m
We present the current state of the DUNE Rucio deployment and plans for extension and expansion. Results will include new monitoring and testing that has been
added as well as plans for moving from the current hybrid system to a native
Rucio deployment.Speaker: Steven Timm (Fermi National Accelerator Lab. (US))
-
15:45
-
16:30
→
17:00
Break 30m
-
17:00
→
18:00
Community ReportsConvener: Mario Lassnig (CERN)
-
17:00
Customizing Rucio at LCLS 15m
This talk will present the speaker's onboarding experience as someone new to Rucio, covering the aspects of using documentation, building and standing up the containers, and configuring the Rucio system. Furthermore, specifics will be discussed in regard to the customization in development to meet the project requirements for LCLS, a free electron laser that produces ultra fast X-ray pulses.
Speaker: Kenny Lo (SLAC National Accelerator Laboratory) -
17:15
IGWN 15mSpeaker: Gabriele Gaetano Fronze' (INFN Torino (IT) and LIGO-Virgo-Kagra Collaboration (US/IT/JP))
-
17:30
FTS: Updates, Direction and Plans 15mSpeaker: Mihai Patrascoiu (CERN)
-
17:45
AAI/Tokens IAM 15mSpeaker: Andrea Ceccanti (Universita e INFN, Bologna (IT))
-
17:00
-
18:00
→
19:00
Panels: Rucio in a non-grid environmentConvener: Cedric Serfon (Brookhaven National Laboratory (US))
-
18:00
Rucio in a non-grid environment panel 1hSpeakers: Alastair Dewhurst (Science and Technology Facilities Council STFC (GB)), Andrea Manzi, David Cameron (University of Oslo (NO)), Ilija Vukotic (University of Chicago (US)), Mario Lassnig (CERN), Oliver Keeble (CERN)
-
18:00
-
15:00
→
15:45
-
-
09:15
→
10:00
KeynoteConvener: Mario Lassnig (CERN)
-
09:15
Pipe Dreams (and Nightmares) 45m
Australia's Academic and Research Network (AARNet) was established in 1989 and is widely regarded as the founder of the Internet in Australia and renowned as the architect, builder and operator of world-class network infrastructure for research and education.
We are Australia's National Research and Education Network (NREN). We connect over one million users—researchers, faculty, staff and students—at institutions across Australia, supporting education and research across a diverse range of disciplines including high energy physics, climate science, genomics, radio astronomy and the arts.
Nationally, AARNet interconnects Australian universities, the CSIRO, and other organisations who have a research and education mission, or with whom the education and research sector interacts. These include hospitals, vocational training providers, schools and museums. Internationally, AARNet interconnects the Australian Research and Education (R & E) community to the world – and continuously develops new capabilities and partnerships to facilitate seamless data access and transfer.
Today, we'll talk about some of the work we've done in the data access and transfer space, challenges we've faced, and our plans for the future.Speaker: Crystal Michelle Chua
-
09:15
-
10:00
→
11:05
Community Reports: WFMSConvener: Cedric Serfon (Brookhaven National Laboratory (US))
-
10:00
Panda 15mSpeaker: Paul Nilsson (Brookhaven National Laboratory (US))
- 10:15
-
10:30
CMS Workload Management and Rucio integration 15mSpeaker: Todor Trendafilov Ivanov (University of Sofia - St. Kliment Ohridski (BG))
-
10:45
Discussion 20m
-
10:00
-
11:05
→
11:25
Break 20m
-
11:25
→
12:25
Community ReportsConvener: Martin Barisits (CERN)
-
11:25
Belle II 15mSpeaker: Cedric Serfon (Brookhaven National Laboratory (US))
-
11:40
Rucio and ScienceMesh: Enabling data management for the CS3 community 15mSpeakers: Giuseppe Lo Presti (CERN), Rahul Chauhan
- 11:55
-
11:25
-
15:00
→
16:40
Community Reports: AstronomyConvener: Rosie Bolton (SKA Organisation)
- 15:00
-
15:05
SKA Rucio deployment and metadata/ data lifecycle use case 15mSpeaker: Rohini Joshi (SKA Organisation)
- 15:20
-
15:35
CTA rucio use case and development with cloud storage 15mSpeaker: Frederic Gillardo (Centre National de la Recherche Scientifique (FR))
-
15:50
LOFAR Use Cases in ESCAPE - Experience and Future Directions 15m
We will be presenting our experiences and results for LOFAR use case using ESCAPE data lake and Rucio. Reflections on future requirements including for other data sets (e.g. APERTIF) shall also be deliberated.
Speaker: Yan Grange (ASTRON) - 16:05
- 16:20
-
16:40
→
17:00
Break 20m
-
17:00
→
18:00
Community ReportsConvener: Eric Vaandering (Fermi National Accelerator Lab. (US))
-
17:00
ESCAPE Data Lake as a Service 15m
Experiments and scientists, whether in the process of designing and building up a data management system or managing multi-petabyte data historically, gather in the European Science Cluster of Astronomy & Particle physics ESFRI research infrastructures (ESCAPE) project to address computing challenges by developing common solutions in the context of the EOSC.
A modular ecosystem of services and tools constitutes the ESCAPE Data Lake, which is exploited by flagship ESFRIs in Astro-particle Physics, Electromagnetic and Gravitational-Wave Astronomy, Particle Physics, and Nuclear Physics to pursue together the FAIR and open-access data principles.
This infrastructure fulfils the needs of the ESCAPE community in terms of data organisation, management, and access, and dedicated assessment exercises demonstrated its robustness.
As a result, collaborating sciences are choosing their reference implementations of the various technologies among the proposed solutions.
A variety of challenges and specific use cases boost ESCAPE to carefully take into account both user and infrastructure perspectives, and contributed to successfully conclude the pilot phase beyond expectations, embarking on a like-production prototype stage.
The ongoing phase of the project aims at consolidating the functionalities of the services, e.g. integrating token-based AuthN/Z or deploying a tailored content delivery and caching layer, and at simplifying the user experience. Specifically for this reason, a considerable effort is being devoted towards a DataLake-as-a-Service whose goal is to provide the end-user with a Notebook ready-to-be-used and fully integrated with the Data Lake.
ESCAPE milestones achieved during the length of the project represent a fundamental accomplishment under both sociological and computing model aspects for different scientific communities that should address upcoming data management and computing challenges in the next decade.Speaker: Dr Riccardo Di Maria (CERN) -
17:15
XENON 15mSpeaker: Paschalis Paschos (University of Chicago)
-
17:30
GO/QoS/MAS 15mSpeaker: Matt Snyder (Brookhaven National Laboratory)
-
17:00
-
18:00
→
19:00
Panels: Astronomy & MetadataConvener: Rosie Bolton (SKA Organisation)
-
18:00
Astronomy & Metadata 1hSpeakers: Cedric Serfon (Brookhaven National Laboratory (US)), Dave Morris, Greg Daues (NCSA), Pandey Vishambhar (ASTRON), Rob Barnsley (SKAO)
-
18:00
-
09:15
→
10:00
-
-
15:00
→
16:00
Community Reports: Long tail of scienceConvener: Alastair Dewhurst (Science and Technology Facilities Council STFC (GB))
-
15:00
Introduction - What is the 'Long Tail of Science'? 10mSpeaker: Alastair Dewhurst (Science and Technology Facilities Council STFC (GB))
-
15:10
Multi-VO Rucio 15mSpeaker: Timothy John Noble (Science and Technology Facilities Council STFC (GB))
- 15:25
-
15:40
Fermilab view 15mSpeaker: Brandon White (Fermi National Accelerator Lab. (US))
-
15:00
-
16:00
→
16:10
Break 10m
-
16:10
→
17:10
Panels: Transfer & StorageConvener: Mario Lassnig (CERN)
-
16:10
Transfer & Storage panel 1hSpeakers: Andrea Ceccanti (Universita e INFN, Bologna (IT)), Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US)), Hannah Short (CERN), Martin Barisits (CERN), Mihai Patrascoiu (CERN), Paul Millar
-
16:10
-
17:10
→
18:00
Opening & Closing: Discussion, Photo and ClosingConvener: Martin Barisits (CERN)
-
17:10
Discussion 30m
-
17:40
Photo 5m
-
17:45
Closing 10m
-
17:10
-
15:00
→
16:00