4th Rucio Community Workshop (Virtual)

Europe/Zurich
Description

Rucio is a software framework that provides functionality to organize, manage, and access large volumes of scientific data using customisable policies. The data can be spread across globally distributed locations and across heterogeneous data centers, uniting different storage and network technologies as a single federated entity. Rucio offers advanced features such as distributed data recovery or adaptive replication, and is highly scalable, modular, and extensible. Rucio has been originally developed to meet the requirements of the high-energy physics experiment ATLAS, and is continuously extended to support LHC experiments and other diverse scientific communities.

We set up a mailing list to which you can subscribe and where we will send more details about the program in the coming weeks.

We also created a Slack channel dedicated to the workshop discussion on the Rucio Slack workspace (Invitation Link). Join #workshop.

Participants
  • Agustin Bruzzese
  • Alastair Dewhurst
  • Aman Goel
  • Andrea Manzi
  • Andrea Rendina
  • Andreas Haupt
  • Andrew Bohdan Hanushevsky
  • Andrew McNab
  • Anil Panta
  • Antoon Prins
  • Arfath Pasha
  • Armin Nairz
  • Artem Petrosyan
  • Barbara Borgia
  • Benedikt Ziemons
  • Brandon White
  • Brian Yanny
  • Catalin Condurache
  • Cecilia Duran Osuna
  • Cedric Serfon
  • Chien-De Li
  • Claudio Cacciari
  • Crystal Michelle Chua
  • Daniela Bauer
  • Dave Morris
  • David Cameron
  • David South
  • DEV MALIK
  • Dimitrios Christidis
  • Dirk Hufnagel
  • Doug Benjamin
  • Erich Birngruber
  • Etienne Lyard
  • Fabio Andrijauskas
  • Fabio Hernandez
  • Federica Legger
  • Fernando GARZON MIGUEZ
  • Frederic Gillardo
  • Gabriele Gaetano Fronze'
  • Gaia Cipolletta
  • Gareth Hughes
  • Garyfallia Paspalaki
  • Gavin Biffar
  • Gernot Maier
  • Gino Marchetti
  • Giuseppe Andronico
  • Giuseppe Lo Presti
  • Gonzalo Merino
  • Greg Daues
  • Heather Kelly
  • I Ueda
  • Ilija Vukotic
  • Ivan Glushkov
  • James Walder
  • JJ Kavelaars
  • Johan AUSSENAC
  • John Steven De Stefano Jr
  • Judith Lorraine Stephen
  • Kajal Dixit
  • Katy Ellis
  • Kenny Lo
  • Konstantinos Servis
  • Lachlan Campbell
  • Lorena Lobato Pardavila
  • Maithili Kalamkar Stam
  • Marco Günter
  • Marek Szuba
  • Mario Lassnig
  • Marshall McDonnell
  • Martin Barisits
  • Matthew Snyder
  • Michelangelo Bottura
  • Michelle Butler
  • Mieke Bouwhuis
  • Muhammad Aleem Sarwar
  • Natalie Danezi
  • Nick Smith
  • Olga Sergijenko
  • Oliver Keeble
  • Onno Zweers
  • Pascal Paschos
  • Patrick Dowler
  • Paul James Laycock
  • Paul Musset
  • Paul Nilsson
  • Peter Caligari
  • Radu Carpa
  • Rahul Chauhan
  • Raymond Oonk
  • Riccardo Bruno
  • Riccardo Di Maria
  • Rizart Dona
  • Rob Barnsley
  • Robert Andrew Currie
  • Robert Illingworth
  • Rohini Joshi
  • Roland Walter
  • Ron Trompert
  • Rosie Bolton
  • Ruslan Mashinistov
  • Sayanta Neogi
  • sayanta neogi
  • Shishir Roy
  • Stefan Piperov
  • Steven Timm
  • Tiansu Yu
  • Tiansu Yu
  • Timothy Noble
  • Tuan Minh Pham
  • V.N. Pandey
  • Vasilka Chergarova
  • Volodymyr Savchenko
  • Wei Yang
  • Wenlong Yuan
  • Wilko Kroeger
  • Yan Grange
  • Yuyi Guo
Videoconference
4th Rucio Community Workshop (Virtual)
Zoom Meeting ID
66591244385
Host
Martin Barisits
Alternative hosts
Mario Lassnig, Cedric Serfon, Alastair Dewhurst
Useful links
Join via phone
Zoom URL
    • 3:00 PM 3:45 PM
      Opening & Closing
    • 3:45 PM 4:30 PM
      Community Reports
      Convener: Alastair Dewhurst (Science and Technology Facilities Council STFC (GB))
      • 3:45 PM
        CMS 15m
        Speaker: Eric Vaandering (Fermi National Accelerator Lab. (US))
      • 4:00 PM
        Rucio for the Light Dark Matter eXperiment (LDMX) 15m

        The Light Dark Matter eXperiment (LDMX) is a planned small-scale accelerator-based experiment to search for dark matter in the sub-GeV mass region. Finalizing the design of the detector relies on Monte-Carlo simulation of expected physics processes. A distributed computing pilot project was initiated based around existing software used by other communities, including Rucio for data management. Rucio is primarily used as a dataset catalog, and LDMX also makes extensive use of Rucio metadata to record physics properties as well as bookkeeping information on the data produced. In this talk we describe how Rucio is used by LDMX and propose some possible extensions to the metadata functionality.

        Speaker: David Cameron (University of Oslo (NO))
      • 4:15 PM
        DUNE Rucio Usage and Status 15m

        We present the current state of the DUNE Rucio deployment and plans for extension and expansion. Results will include new monitoring and testing that has been
        added as well as plans for moving from the current hybrid system to a native
        Rucio deployment.

        Speaker: Steven Timm (Fermi National Accelerator Lab. (US))
    • 4:30 PM 5:00 PM
      Break 30m
    • 5:00 PM 6:00 PM
      Community Reports
      Convener: Mario Lassnig (CERN)
      • 5:00 PM
        Customizing Rucio at LCLS 15m

        This talk will present the speaker's onboarding experience as someone new to Rucio, covering the aspects of using documentation, building and standing up the containers, and configuring the Rucio system. Furthermore, specifics will be discussed in regard to the customization in development to meet the project requirements for LCLS, a free electron laser that produces ultra fast X-ray pulses.

        Speaker: Kenny Lo (SLAC National Accelerator Laboratory)
      • 5:15 PM
        IGWN 15m
        Speaker: Gabriele Gaetano Fronze' (INFN Torino (IT) and LIGO-Virgo-Kagra Collaboration (US/IT/JP))
      • 5:30 PM
        FTS: Updates, Direction and Plans 15m
        Speaker: Mihai Patrascoiu (CERN)
      • 5:45 PM
        AAI/Tokens IAM 15m
        Speaker: Andrea Ceccanti (Universita e INFN, Bologna (IT))
    • 6:00 PM 7:00 PM
      Panels: Rucio in a non-grid environment
      Convener: Cedric Serfon (Brookhaven National Laboratory (US))
      • 6:00 PM
        Rucio in a non-grid environment panel 1h
        Speakers: Alastair Dewhurst (Science and Technology Facilities Council STFC (GB)), Andrea Manzi, David Cameron (University of Oslo (NO)), Ilija Vukotic (University of Chicago (US)), Mario Lassnig (CERN), Oliver Keeble (CERN)
    • 9:15 AM 10:00 AM
      Keynote
      Convener: Mario Lassnig (CERN)
      • 9:15 AM
        Pipe Dreams (and Nightmares) 45m

        Australia's Academic and Research Network (AARNet) was established in 1989 and is widely regarded as the founder of the Internet in Australia and renowned as the architect, builder and operator of world-class network infrastructure for research and education.
        We are Australia's National Research and Education Network (NREN). We connect over one million users—researchers, faculty, staff and students—at institutions across Australia, supporting education and research across a diverse range of disciplines including high energy physics, climate science, genomics, radio astronomy and the arts.
        Nationally, AARNet interconnects Australian universities, the CSIRO, and other organisations who have a research and education mission, or with whom the education and research sector interacts. These include hospitals, vocational training providers, schools and museums. Internationally, AARNet interconnects the Australian Research and Education (R & E) community to the world – and continuously develops new capabilities and partnerships to facilitate seamless data access and transfer.
        Today, we'll talk about some of the work we've done in the data access and transfer space, challenges we've faced, and our plans for the future.

        Speaker: Crystal Michelle Chua
    • 10:00 AM 11:05 AM
      Community Reports: WFMS
      Convener: Cedric Serfon (Brookhaven National Laboratory (US))
    • 11:05 AM 11:25 AM
      Break 20m
    • 11:25 AM 12:25 PM
      Community Reports
      Convener: Martin Barisits (CERN)
    • 3:00 PM 4:40 PM
      Community Reports: Astronomy
      Convener: Rosie Bolton (SKA Organisation)
    • 4:40 PM 5:00 PM
      Break 20m
    • 5:00 PM 6:00 PM
      Community Reports
      Convener: Eric Vaandering (Fermi National Accelerator Lab. (US))
      • 5:00 PM
        ESCAPE Data Lake as a Service 15m

        Experiments and scientists, whether in the process of designing and building up a data management system or managing multi-petabyte data historically, gather in the European Science Cluster of Astronomy & Particle physics ESFRI research infrastructures (ESCAPE) project to address computing challenges by developing common solutions in the context of the EOSC.
        A modular ecosystem of services and tools constitutes the ESCAPE Data Lake, which is exploited by flagship ESFRIs in Astro-particle Physics, Electromagnetic and Gravitational-Wave Astronomy, Particle Physics, and Nuclear Physics to pursue together the FAIR and open-access data principles.
        This infrastructure fulfils the needs of the ESCAPE community in terms of data organisation, management, and access, and dedicated assessment exercises demonstrated its robustness.
        As a result, collaborating sciences are choosing their reference implementations of the various technologies among the proposed solutions.
        A variety of challenges and specific use cases boost ESCAPE to carefully take into account both user and infrastructure perspectives, and contributed to successfully conclude the pilot phase beyond expectations, embarking on a like-production prototype stage.
        The ongoing phase of the project aims at consolidating the functionalities of the services, e.g. integrating token-based AuthN/Z or deploying a tailored content delivery and caching layer, and at simplifying the user experience. Specifically for this reason, a considerable effort is being devoted towards a DataLake-as-a-Service whose goal is to provide the end-user with a Notebook ready-to-be-used and fully integrated with the Data Lake.
        ESCAPE milestones achieved during the length of the project represent a fundamental accomplishment under both sociological and computing model aspects for different scientific communities that should address upcoming data management and computing challenges in the next decade.

        Speaker: Dr Riccardo Di Maria (CERN)
      • 5:15 PM
        XENON 15m
        Speaker: Paschalis Paschos (University of Chicago)
      • 5:30 PM
        GO/QoS/MAS 15m
        Speaker: Matt Snyder (Brookhaven National Laboratory)
    • 6:00 PM 7:00 PM
      Panels: Astronomy & Metadata
      Convener: Rosie Bolton (SKA Organisation)
      • 6:00 PM
        Astronomy & Metadata 1h
        Speakers: Cedric Serfon (Brookhaven National Laboratory (US)), Dave Morris, Greg Daues (NCSA), Pandey Vishambhar (ASTRON), Rob Barnsley (SKAO)
    • 3:00 PM 4:00 PM
      Community Reports: Long tail of science
      Convener: Alastair Dewhurst (Science and Technology Facilities Council STFC (GB))
    • 4:00 PM 4:10 PM
      Break 10m
    • 4:10 PM 5:10 PM
      Panels: Transfer & Storage
      Convener: Mario Lassnig (CERN)
      • 4:10 PM
        Transfer & Storage panel 1h
        Speakers: Andrea Ceccanti (Universita e INFN, Bologna (IT)), Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US)), Hannah Short (CERN), Martin Barisits (CERN), Mihai Patrascoiu (CERN), Paul Millar
    • 5:10 PM 6:00 PM
      Opening & Closing: Discussion, Photo and Closing
      Convener: Martin Barisits (CERN)
      • 5:10 PM
        Discussion 30m
      • 5:40 PM
        Photo 5m
      • 5:45 PM
        Closing 10m