Rucio Meeting

Thursday 5 Oct 2023, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

1 News

Release names 2024.png

• October release schedule
  • 32.? Oct 9
  • 32.? Oct 23 will possibly be moved to Oct 30
• Release names 2024 vote
  • Rucio 34 "Donkey Potter and the Data Cache"
  • Rucio 35 LTS "Donkey and the Data Factory"
  • Rucio 36 "Donkey Unchained"

2 Community News & DevOps roundtable

• ATLAS
• Fermilab DUNE/...
  • RUBIN
    • transmogrifier does not work on dev instance
      • log shows new dids being processed/noticed
      • Does not generate any rules though
        • Duplicate rule errors
  • Figuring WebUI out as well
• Belle II
• ESCAPE/EOSC-Future
  • Incoming PR for Jupyterlab Extension
    • Fixing tests
    • Updating to the newest version
  • Updating documentation
• DUNE/Edinburgh
  • policy package updates
    • Adding checks for did presence
      • datasets, files
  • Coming to auditor for object stores soon
  • Creating VO-specific tests for DUNE
• ALTO
  • Good progress for WebUI/Transfer monitoring
    • Working on backend now
    • Discussion about topology view for transfer monitoring
  • In parallel working on sorting algorithms
• MultiVO/RAL

3 Developers roundtable

Progress-2023-10-05.png

• Rucio 33 "Eternal Sunshine of the Donkey's Mind" roadmap
  • In Progress
    • Design and implement a token cache module #6183 [Dimitrios]
    • Introduce the RSE decommissioner daemon #6321 [Yutaro, Dimitrios]
      • One query under discussion, otherwise underway and should be included
    • WebUI Release 33 page tracker (4/16) #301 [Mayank, Anton, Eraldo]
      • Handover Anton -> Eraldo
      • Mayank working on deployment side of things
    • Transfer status visual representation #6189 [Radu, Jensen]
      • Topology view it depends on third-party dependencies and their integration
      • Historical transfer mechanism in preparation
        • When done the data can be visualized in webui
    • Dynamic distance between RSEs #5012 [Radu, Jensen]
      • In early stages
      • Needs more discussion
      • Potentially not for 33
    • revisit submitter source selection strategy #5776 [Radu, Jensen]
      • Idea is somebody from Yale Team can implement it
  • In Review
    • Unable to Delete File DID via Undertaker #5154 [Anton]
      • --> potentially scratch for 33
    • Incorporate IAM-based testing for token workflows #6188 [Eraldo, Dimitrios]
      • Eraldo prepared the container
        • Can check with curl requests if it is "good enough"
        • Full testing (via rucio code) - code does not exist in Rucio yet
    • scitags/packet marking support #5856 [Radu]
      • PR is open and needs review
      • Need final decision from FTS side about the final parameter name for the tag
  • Todo
    • Design and implement a token request module #6187 [Dimitrios]
      • Will definitely come
        • Probably not as flexible as needed down the road (Single profile, single config)
    • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
      • Will come
    • foreign key error on deleting dids in reaper #5733 [Martin]
      • Potentially drop
    • Review state of temporary dids (and thus Light Reaper) #6317 [Dimitrios]
      • Executive decision: Let's take it out
    • Tests cleanup [Radu] #6212
      • Update ticket
  • Done
    • Review lost data handling (replica-recoverer) [Martin]
      • Open an issue from the discussion [Cedric] --> #6337
• Documentation corner
  • Documentation and dev guidelines for Mypy type annotations #116 [Mayank, Martin]
  • Document environmental variables affecting the client #171 [Dimitrios]
  • Improve documentation on rucio.cfg vs configuration table #183 [Radu]
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Add instruction about DB partitioning #185 [Martin]
  • bridge the gap between running rucio in demo env and full production deployment #187 [Radu]
  • Introduce documentation on subscriptions #190 [Cedric]
  • list_file replicas usage #6333 [Eraldo]
• Other topics

4 AOB