Discussion of FCC Dataset Management System

Monday 8 Sept 2025, 10:00 → 11:30 Europe/Zurich

32/S-C22 (CERN)

Minutes from Discussion of FCC Dataset Management System

Present: Andre, Birgit, Brieuc, Giovanni, Jan, Juraj, Louis

• Rucio + DIRAC are still not enough to cover all needs
  • In terms of metadata
  • ATLAS has additional AMI service to provide detailed metadata
• No full metadata catalog in Rucio
  • Plugin can add this functionality
• In case of Rucio we would need to come up with a dataset naming convention
  • DIRAC uses LFNs
• Rucio provided as a CERN IT service for small experiments
  • No big hiccups encountered during the Pilot period
  • Not all actually started using it
• Rucio containers can be nested
  • In newer versions there can "one line address"
• Users are required to have x509 or token
  • This rises a bar for the common analyzer to join
  • Work under way to integrate with CERN Keykloak (Auth service)
• Dataset relations are present in Rucio only as the file name/metadata
  • Dirac file catalog has it build in
  • We need provenance graphs when the data sample production will get more complex (FCC-hh)
• FTS handles the Rucio data transfers
  • Dirac has plugin for it as well
• We agreed to set up CNAF SE as Rucio SE and investigate the usage and needs
• Validation capabilities boil down to metadata fields
• Monitoring tools need to be set up separately and then brought to Rucio
• Dirac DMS and Rucio are very similar capabilities wise
  • DIRAC can have diverse set of plugin
  • DIRAC can also be described as declarative
• User production to be moved fully to DIRAC
  • No need for personal EventProducer
  • We need to have user production process as close to the real production as possible
  • Dirac user job can output to a common scratch disk and Rucio will pick the samples there (and turn them into official production)
• Physics dataset validation is handled only by the MC production manager (Louis)
  • We need more people to join the effort
  • Can be handled in the FCC Physics website by metadata flag
  • The physics validation started to be developed at the PF workshop last week, not operational yet

10:00 → 10:20 Current Status

Speaker: Juraj Smiesko (CERN)

FCC Dataset Management: Current Status

10:20 → 10:40 Rucio Showcase

Speaker: Giovanni Guerrieri (CERN)

Rucio4FCC