Rucio Meeting

Thursday 17 Oct 2024, 15:00 → 16:00 Europe/Zurich

Dimitrios Christidis (CERN)

15:00 → 15:05 News

• 7th Rucio Community Workshop
  • Thanks to everyone who participated!
  • Recordings are available
  • Please consider responding to the short anonymous survey about the programme
• October release plan
  • Oct-14 35.4.1
  • Oct-28 35.?
• 36 “Donkey Unchained”
  • End of November

15:05 → 15:25 Community News & DevOps roundtable

• ATLAS
  • Tape metadata
    • To be tested before the heavy-ion run, to run during
  • Poor deletion perfomance
    • Could be related #6512
    • We halve chunk size to 500 prelicas
  • (old) Web UI issues (running on Kubernetes)
    • Lots of dropped requests
      • Can be seen by connected into the pod
      • AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    • Due to the Apache server configuration limitation
    • Solved by doubling the amount of UI pods
    • CMS commented that have received similar complaints
• CMS
  • BB8 daemon
    • Works on dataset locks, but doesn’t properly take into account rules on containers, thus creating multiple replication rules
    • Followed up in issue #7109, PR available #7124
• DUNE
  • State of PR #6563?
    • Blocked by #6667
  • Meeting between DUNE, Rucio, and FTS to discuss tokens with CILogon
    • FTS test script is triggering a bug in CILogon
    • Client credentials flow missing, but being worked on
      • In next release™
• Fermilab
  • Upgraded all instances to release 35.4.0
  • Enabled the new Web UI
    • Issue with resource constraint
• ESCAPE
• ALTO
• CERN IT
  • Will look into the Reaper issue
  • Backporting multi-hop PR
• Multi-VO

15:25 → 15:55 Developers roundtable

• 36 "Donkey Unchained" roadmap
  • Todo
    • factorize duplicate messaging code into a common module or class #6423 [Alex]
    • Allow for token providers other than IAM #6630 [Dimitrios]
    • Create a new API endpoint for the clients to request tokens #6638 [Dimitrios]
    • Implement new token authentication for download #7029 [Dimitrios]
    • Prevent multiple Reaper threads from working on the same replicas #6512[Hugo]
    • Improve docstrings #6628 [Riccardo]
    • Evaluate archive workflows for rules (Using an archive as a source) #6643[Riccardo]
  • In Progress
    • Make new token support Multi-VO compatible #6406 [Dimitrios]
    • Review operations where SSL verification is disabled #6632 [Dimitrios]
      • Draft PR for split into two parts
    • Retire RSE limits #6629 [Erling, Dimitrios]
    • Prepare replacement of current policy import with policy packages #4798 [James, Martin]
      • Belle II policy -> check with Cedric [on Martin]
      • ATLAS policy -> is there, but needs to be deployed
        • Need to identify a best-practice to deploy a policy package
    • WebUI Tracker 36 [Mayank, Viktoria]
      • 35.2 release published
      • Streaming issues fixed
      • 35.3 this week or early next week, including dashboard view and full create rule workflow
      • X509 auth still open
    • Refactor all ATLAS-related code into ATLAS policy package #7027 [Riccardo]
    • Better define unit tests and integ tests #7007 [Riccardo]
    • Improve test coverage #7008 [Riccardo]
  • In Review
    • Allow permissions to optionally return a message #6580 [James]
      • PR #7001 approved
    • Re-write and standardization of client command structure #6639 [Maggie]
      • Please review! (Need comments from Dimitrios, Mario, Riccardo)
  • Done
  • Delayed
    • Make WebDAV protocols on par with GFAL #6633 [Dimitrios]
• Documentation corner
  • Document activities used by rucio-daemons #326 [???]
  • Generate "Configuration parameters" documentation page automatically, to avoid mismatches between documentation and code #325 [???]
  • Error when trying to upload file with custom name containing slashes #336 [???]
  • Make a table for all daemon and its functionality #296 [Hugo]
  • Documentation (developer): Add guidelines for testing API-related changes #322 [???]
  • Documentation for RSE settings #293 [???]
  • Document how deletion occurs. #288 [Anil]
    • PR #295 underway
  • Introduce documentation on subscriptions #190 [Cedric]
  • bridge the gap between running rucio in demo env and full production deployment #187 [Hugo, Giovanni]
    • PR #349 merged.
  • Add instruction about DB partitioning #185 [Martin]
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Document environmental variables affecting the client #171 [Dimitrios]
    • PR #366 merged.
  • Changing auth_type should invalidate current Rucio authn token #321 [???]
• Other topics

15:55 → 16:00 AOB

• Will have a presentation about the new Web UI next week by Victoria.