Rucio Meeting

Thursday 12 Sept 2024, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• 1.29 LTS release now in EOL
  • No longer supported
• September release schedule
  • Sep-09 35.3.0
  • Sep-16 32.8.3 LTS
  • Sept-23 35.?
• Rucio Workshop
  •  Please register!

15:05 → 15:25 Community News & DevOps roundtable

• ATLAS
  • Long-lived token test still running
• CMS
  • DB Issue with COLLECTION_REPLICA update job
    • Queue-Table got too big, job cannot manage it anymore
    • For now, clean out the table, re-sync with a manual job
    • In the medium term: Need to optimise the CMS job (similar to the ATLAS one)
    • In the long term: collection_replicas will go away
  • Running 35.2.1
  • Overwrite replicas error message changed in FTS 3.12.12, PR to check for new error message in Rucio
• Fermilab / DUNE / RUBIN
• DUNE
  • CI Community tests
    • Is authentication being used?
      • Yes, but right now possibly only x509, IAM container prepared, not used yet
  • CILogon
    • When should the work start?
      • Right now, James is already looking into it
  • Dark data identification and cleanup
    • ATLAS workflow: is it documented? 
      • Mostly follow the auditor code, documention, not so much
      • Code mostly self-explenatory
• DaFab
  • Exploring new metadata functionality and improvements to bring to Rucio
• ESCAPE
  • VRE Update
• CERN IT
  • Hiring to work on open-data
• ALTO

15:25 → 15:55 Developers roundtable

Progress-2024-09-12.png

• 36 "Donkey Unchained" roadmap
  • Todo
    • factorize duplicate messaging code into a common module or class #6423 [Alex]
    • Allow for token providers other than IAM #6630 [Dimitrios]
    • Create a new API endpoint for the clients to request tokens #6638 [Dimitrios]
    • Implement new token authentication for download #7029 [Dimitrios]
    • Make WebDAV protocols on par with GFAL #6633 [Dimitrios]
    • Prevent multiple Reaper threads from working on the same replicas #6512 [Hugo]
    • Re-write and standardization of client command structure #6639 [Maggie]
    • Improve docstrings #6628 [Riccardo]
    • Evaluate archive workflows for rules (Using an archive as a source) #6643 [Riccardo]
  • In Progress
    • Make new token support Multi-VO compatible #6406 [Dimitrios]
    • Review operations where SSL verification is disabled #6632 [Dimitrios]
    • Retire RSE limits #6629 [Erling, Dimitrios]
    • Prepare replacement of current policy import with policy packages #4798 [James, Martin]
    • WebUI Tracker 36 [Mayank, Viktoria]
      • Background thread not getting enough resources -> fixed
      • X509 auth -> Being tested (Firefox, Chrome, Safari)
      • Many smaller issues
    • Refactor all ATLAS-related code into ATLAS policy package #7027 [Riccardo]
    • Better define unit tests and integ tests #7007 [Riccardo]
    • Improve test coverage #7008 [Riccardo]
  • In Review
    • Allow permissions to optionally return a message #6580 [James]
      • PR #7001 approved.
  • Done
  • Delayed
• Documentation corner
  • Document activities used by rucio-daemons #326 [???]
  • Generate "Configuration parameters" documentation page automatically, to avoid mismatches between documentation and code #325 [???]
  • Error when trying to upload file with custom name containing slashes #336 [???]
  • Make a table for all daemon and its functionality #296 [Hugo]
  • Documentation (developer): Add guidelines for testing API-related changes #322 [???]
  • Documentation for RSE settings #293 [???]
  • Document how deletion occurs. #288 [Anil]
  • Introduce documentation on subscriptions #190 [Cedric]
  • bridge the gap between running rucio in demo env and full production deployment #187 [Hugo, Giovanni]
    • PR #349 approved but not yet merged.
  • Add instruction about DB partitioning #185 [Martin]
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Document environmental variables affecting the client #171 [Dimitrios]
    • PR #366 approved but not yet merged.
  • Changing auth_type should invalidate current Rucio authn token #321 [???]
• Other topics
  • Onging work on Jupyterlab-Plugin
    • Search behaviour

15:55 → 16:00 AOB