Rucio Meeting

Thursday 15 Feb 2024, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• Data Challenge 24
  • Rucio 33.5.0 release

15:05 → 15:25 Community News & DevOps roundtable

• ATLAS
  • Production everything fine
  • DC24: Looking OK as well++
    • No particular problems in Rucio
    • Network config wise, tokens, etc. need some special attention
  • Globus might will become more important due to Policy change @ NERSC
• CMS
  • Prod & Int moved to 33.5.0
  • Running smooth similar to ATLAS
  • #6396 and #6360 waiting for feedback 
  • Suspended rules #6434
    • Wait until Eric is back and have offline discussion
• Fermilab Rucio DUNE, RUBIN, etc
  • Fermilab 33.5.0 
  • RUBIN 33.4.0.post1
  • DUNE getting ready for DC24
  • mu2e
    • Atomic registration of larger sets of files which was discussed before no longer needed
  • list-rules --account duneprod issue (60k+ rules) fixed after increasing pod cpu+mem
    • Pods were OOMKilled, increasing CPU fixed it
• DUN
  • 1.29 -> 33.4.0 upgrade triggered bug in DUNE policy package
    • Validation important with major release upgrades
  • Collection replicas report 0 number of files (--long produces wrong results)
  • Globus
    • NERSC (Used by DUNE, ATLAS, CMS)
      • Had an xrootd endpoint, however security model changed and this is not longer usable
      • Need to integrate Globus fully to Rucio (Integration tests + validation)
• Belle II
  • DC24 good

15:25 → 15:55 Developers roundtable

Progress-2024-02-15.png

• Rucio 34 "Donkey Potter and the Data Cache" roadmap
  • In Progress
    • foreign key error on deleting dids in reaper #5733 [Alex]
      • Mostly a conceptual discussion -> Discussion with Martin
    • factorize duplicate messaging code into a common module or class #6423 [Alex]
    • Deployment and Release Workflow #401 [Mayank, Eraldo]
      • Blocker with X509 auth
      • Need some upstream changes in rucio for multi-account support
      • Need a change in helm-charts
    • Missing WebUI Release 33 page tracker #301 [Mayank, Eraldo]
    • Migrate Dashboard to Clean Architecture #158 [Mayank, Eraldo]
    • Unable to Delete File DID via Undertaker #5154 [Riccardo]
      • Refactoring of daemons first, Review being adressed now
    • Type annotate the code #6454 [Riccardo]
      • Pushed first PR for this
    • Update extension for v32 (and higher) compatibility #25 [Francesc, Enrique]
      • Minor documentation issues being worked on
  • In Review
    • Continue migration to SQLAlchemy 2.0 syntax #6057 [Erling]
      • 5 PRs due to size (Two are submitted upstream, but other 3 can be pushed)
    • Refactor policy package algorithm code #6382 [James]
    • Metadata for tape co-location and transfer prority #6398 [Maggie]
    • Update/Re-design core.meta module #5224 [Maggie, Rob]
  • Todo
    
    • bridge the gap between running rucio in demo env and full production deployment #187 [Radu, Enrique]
  • Done
    • Add Token based TPC tests to the CI #6451 [Radu]
  • Delayed
• Documentation corner
  • Documentation and dev guidelines for Mypy type annotations #116 [Mayank, Martin]
  • Document environmental variables affecting the client #171 [Dimitrios]
  • Improve documentation on rucio.cfg vs configuration table #183 [Radu]
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Add instruction about DB partitioning #185 [Martin]
  • bridge the gap between running rucio in demo env and full production deployment #187 [Radu]
  • Introduce documentation on subscriptions #190 [Cedric]
  • WebUI: Improve Docs #255 [Eraldo]
  • Add instructions for Mac Apple Silicon in the developer section #261 [Eraldo]
    • Under Review - Comments posted, needs iteration
  • Add Rucio QoS RSE description and instructions #268 [Matt]
    • Under Review - Comments posted, needs iteration
• Other topics
  •  

15:55 → 16:00 AOB