Rucio Meeting

Thursday 12 Oct 2023, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

1 News

• Rucio workshop next week
  • No Rucio meeting!

2 Community News & DevOps roundtable

• ATLAS
  • Effort to run Rucio instances with ipv6 enabled
    • Not trivial @CERN, especially for deletions
• CMS
  • Auth servers still on 1.31 due to alma9 certificate issues
  • New person joining FNAL & CMS team!
• Fermilab / RUBIN / mu2e / etc
  • mu2e
    • Starting to interface with Rucio API
  • FNAL Cybersecurity asked to disable track/trace methods on WebUI
    • serverstatus etc
    • To turn off in containers by default?
• ESCAPE / EOSC
  • Reaper not correctly deleting files
    • After setting to greedy all fine
  • JupyterLab extension
    • Needs some major changes
    • Fixed tests
    • But with jupyterlab v4 needs some major changes
• ALTO
  • Starting work on distribution trees (for transfers)
    • Please share public branch
  • Integrate visualization library to webui
  • Request tables & statistics need some tests
• KM3Net
  • Busy adding elements / attaching storage
  • Now on upload/injection workflows
  • Different ways how to organise data with Rucio

3 Developers roundtable

Progress-2023-10-12.png

• Rucio 33 "Eternal Sunshine of the Donkey's Mind" roadmap
  • In Progress
    • Design and implement a token cache module #6183 [Dimitrios]
    • Introduce the RSE decommissioner daemon #6321 [Yutaro, Dimitrios]
      • Some development still necessary
    • WebUI Release 33 page tracker (4/16) #301 [Mayank, Anton, Eraldo]
      • Created endpoints for rules pages, create_rule needs more time, as it is more complex
      • Making progress on the webui(v2) container
      • Afterwards moving on to helm-charts
      • StreamingPipeline will need an update
    • Transfer status visual representation #6189 [Radu, Jensen]
      • Gateway and usecase model implemented
      • Library to be integrated G6
        • Need to validate library, from a maintainability, security and UI PoV
    • Dynamic distance between RSEs #5012 [Radu, Jensen]
    • revisit submitter source selection strategy #5776 [Radu, Jensen]
    • Simplify the test pipeline [Radu] #6212
      • Open first PR, several commits
  • In Review
    • Review state of temporary dids (and thus Light Reaper) #6317 [Dimitrios]
      • PR is there, one test is failing -> needs checking
      • Integration tests will fail always once this is merged
        • Push new dev container should fix it, but it did not
  • Todo
    • Design and implement a token request module #6187 [Dimitrios]
    • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
  • Done
    • Incorporate IAM-based testing for token workflows #6188 [Eraldo, Dimitrios]
    • Review lost data handling (replica-recoverer) [Martin]
    • scitags/packet marking support #5856 [Radu]
• Documentation corner
  • Documentation and dev guidelines for Mypy type annotations #116 [Mayank, Martin]
  • Document environmental variables affecting the client #171 [Dimitrios]
  • Improve documentation on rucio.cfg vs configuration table #183 [Radu]
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Add instruction about DB partitioning #185 [Martin]
  • bridge the gap between running rucio in demo env and full production deployment #187 [Radu]
  • Introduce documentation on subscriptions #190 [Cedric]
  • list_file replicas usage #6333 [Eraldo]
• Other topics
  • Cloud storage usage in the community?
    • CMS, none (some ceph with xroot in front)

4 AOB