Rucio Meeting

Thursday 30 Nov 2023, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• jsonschema-specifications issue
  • Affects 32.6.0, possibly 33.0.0rc1 as well
  • Not direct dependency of Rucio, but a dependency of a dependency
    • Once updated, consumes order of magnitude more CPU
  • Under investigation
• 33 Code freeze
  • 33.0.0rc1 released
  • 33.0.0 next week
• SIG Token Update
  • Planned for Dec 14
• December meeting schedule
  • Dec 7 - 33 Eternal Sunshine of the Donkey's Mind Release Retrospective
  • Dec 14 - SIG Token Update
  • Dec 21 - 34 Donkey Potter and the Data Cache Release Roadmap planning
  • Dec 28 - 🎄No meeting

15:05 → 15:25 Community News & DevOps roundtable

• ATLAS
  • Tokens
    • First Rucio injected capability-token FTS transfer!
    • Deletion with capability-tokens worked as well!
    • Submission all good, querying tokens not
      • Not able to mix x509 querying with token transfers
      • FTS & Rucio team working on a fix
  • 33.0.0rc1 on INT
  • 32.6.0 on PROD
    • however, tracing servers on 32.5.1 (see news)
• CMS
  • INT cluster on 32.6.0, planning for 32.6 on PROD
  • Evaluating replica-recoverer daemon
    • Issue to make it VO-agnostic (ATLAS specific rse_expression)
• Fermilab *
  • Upgrading every cluster to 32
  • Working on containers with policy packages
  • Working on issue with scope_name regexp for RUBIN
    • Not working for automatix
      • Worked for 1.29, but not since
• DUNE
  • Update to 32.0
  • Load issues, probably need to go into loadbalancing httpd servers
    • Check with cluster administrators what the best way is
  • IAM vs CILogon
    • Request to add CILogon milestones to the development roadmap
      • Will be post DC-24, before there is only hope ;-)
• ESCAPE
  • 32 webui
• DUNE/Edinburgh
  • PR for auditor w/ object stores
    • Fabio on review
  • Move policy-package algorithms to common?
    • James will have a look on the situation
• ALTO
  • Update webui frontend to test with rc release
    • new-webui container not on dockerub (Mayank: only comes next week, in the meanwhile use webui dev env)
  • Dynamic distance based on ALTO-server metrics

15:25 → 15:55 Developers roundtable

Progress-2023-11-30.png

• Rucio 33 "Eternal Sunshine of the Donkey's Mind" roadmap
  • In Progress
    • Introduce the RSE decommissioner daemon #6321 [Yutaro, Dimitrios]
      • Hopeful, but to be balanced with token-tpc work
    • WebUI Release 33 page tracker (4/16) #301 [Mayank, Eraldo]
      • Needs to be pushed by 1w; some parts with state-changes & reloads trigger infinite updates, needs to be fixed
      • Eraldo busy due to return to institute
  • In Review
  • Todo
  • Done
    • Design and implement a token cache module #6183 [Dimitrios]
    • Design and implement a token request module #6187 [Dimitrios]
    • Incorporate IAM-based testing for token workflows #6188 [Eraldo, Dimitrios]
    • Review state of temporary dids (and thus Light Reaper) #6317 [Dimitrios]
    • Review lost data handling (replica-recoverer) [Martin]
    • scitags/packet marking support #5856 [Radu]
    • Simplify the test pipeline [Radu] #6212
    • revisit submitter source selection strategy #5776 [Radu, Jensen]
    • Transfer status visual representation #6189 [Radu, Jensen]
  • Delayed
    • foreign key error on deleting dids in reaper #5733 [Martin, Cedric]
    • Unable to Delete File DID via Undertaker #5154 [Martin]
    • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
    • Dynamic distance between RSEs #5012 [Radu, Jensen]
      • Problem is to validate any of the strategies; thus the "barrier" to actually use it, is high
      • Everything which has to be merged for the release is DONE
        • What remains is deciding on metrics, activating strategies etc.
• Documentation corner
  • Documentation and dev guidelines for Mypy type annotations #116 [Mayank, Martin]
  • Document environmental variables affecting the client #171 [Dimitrios]
  • Improve documentation on rucio.cfg vs configuration table #183 [Radu]
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Add instruction about DB partitioning #185 [Martin]
  • bridge the gap between running rucio in demo env and full production deployment #187 [Radu]
  • Introduce documentation on subscriptions #190 [Cedric]
  • Versioning Issue in generate_release_notes_index.py #247 [Anton]
  • WebUI: Improve Docs #255 [Eraldo]
  • Add instructions for Mac Apple Silicon in the developer section #261 [Eraldo]
  • Add Rucio QoS RSE description and instructions #268 [Matt]
• Other topics
  •  

15:55 → 16:00 AOB