Rucio Meeting

Thursday 7 Nov 2024, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• November release schedule
  • 35.? Nov-18
  • 36 CODE FREEZE Nov-22
  • 36.0.0rc1 Nov-27
  • 36.0.0 FINAL Dec-04
• DocuSprint 2024 Dec-02 to Dec-13
  • Please sign up

15:05 → 15:25 Community News & DevOps roundtable

• ATLAS
  • Xuantong joining to work on Load Injector Tool
• CMS
  • Finishers killed for OOM in the last week
    • Limited to 1GB, increased to 3GB (Frequency decreased, but still happens)
      • -> Create ticket with traces (#threads)
• Fermilab DUNE/RUBIN/...
• DaFab
• ESCAPE
  • Fixing OIDC issues on cluster
• ALTO
  • Finishing new source selection based on distances
• RAL / MultiVO
  • Upgraded cluster to 35

15:25 → 15:35 SIG Tokens monthly update

Speaker: Dimitrios Christidis (CERN)

• Will now do a monthly update on Token Developments
  • Objective: Integration of CiLogon in Rucio (and FTS)
    • Small meeting between several people Doug, Mihai, James, Steve, Dimitrios, Martin
  • Will have in the first Rucio meeting of each month an update on this activity
  • Right now:
    • Gave Doug, Steve, James a list of commands (REST Calls) which are done betweeen Rucio -> IAM 
      • These need to be exercised to CiLogon as well
      • Ongoing activity
• Noteable meetings:
  
  • Token Trust & Traceability WG October 2024 Call
    • Will have traceability functionality of tokens in Rucio as well
  • WLCG Open Technical Forum (OTF) #1
    • Update on progress/plans on Tokens in Rucio, ATLAS, CMS
• Update on CiLogon testing
  • Identified some developments/changes which need to be done in CiLogon, will be adressed in a future release

15:35 → 15:55 Developers roundtable

Progress-2024-11-07.png

• 36 "Donkey Unchained" roadmap
  • Todo
    • Implement new token authentication for download #7029 [Dimitrios]
      • Depend on #6638, but is also top priority
    • Prevent multiple Reaper threads from working on the same replicas #6512[Hugo]
  • In Progress
    • factorize duplicate messaging code into a common module or class #6423 [Alex]
    • Allow for token providers other than IAM #6630 [Dimitrios]
      • Depend on #6406
    • Make new token support Multi-VO compatible #6406 [Dimitrios]
    • Create a new API endpoint for the clients to request tokens #6638 [Dimitrios]
      • Will be experimental, but is top-priority
    • Review operations where SSL verification is disabled #6632 [Dimitrios]
      • Almost done, just some doc missing
    • Retire RSE limits #6629 [Erling, Dimitrios]
    • Prepare replacement of current policy import with policy packages #4798 [James, Riccardo, Martin]
      • Belle II policy almost DONE
        • Created a temporary policy package for belle 2 for VO tests: 

          https://github.com/rucio/temporary-belle2-policy-package

      • ATLAS policy DONE
    • WebUI Tracker 36 [Mayank, Viktoriaa]
      • Looking into Experiment Logo injects to display in WebUI
      • Page refactoring, only a few left
      • X509 for firefox patch will be in next release
    • Refactor all ATLAS-related code into ATLAS policy package #7027 [Riccardo]
    • Better define unit tests and integ tests #7007 [Riccardo]
      • Few PRs pending review for this
    • Improve test coverage #7008 [Riccardo]
      • Few PRs pending review for this
  • In Review
    • Re-write and standardization of client command structure #6639 [Maggie]
      • Let's merge, see what comments we get (It's backwards compatible anyway)
      • replica subcommand might need some updates
      • Will need some review of the implementation, most feedback/focus was on the command structure/syntax
      • Is not an experimental feature, since it will become the default way/structure of interacting with Rucio!
  • Done
    • Allow permissions to optionally return a message #6580 [James]
  • Delayed
    • Make WebDAV protocols on par with GFAL #6633 [Dimitrios]
    • Improve docstrings #6628 [Riccardo]
    • Evaluate archive workflows for rules (Using an archive as a source) #6643[Riccardo]
• Documentation corner
  • Document activities used by rucio-daemons #326 [???]
  • Generate "Configuration parameters" documentation page automatically, to avoid mismatches between documentation and code #325 [???]
  • Error when trying to upload file with custom name containing slashes #336 [???]
  • Make a table for all daemon and its functionality #296 [Hugo]
  • Documentation (developer): Add guidelines for testing API-related changes #322 [???]
  • Documentation for RSE settings #293 [???]
  • Document how deletion occurs. #288 [Anil]
    • PR #295 underway
  • Introduce documentation on subscriptions #190 [Cedric]
  • Add instruction about DB partitioning #185 [Martin]
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Changing auth_type should invalidate current Rucio authn token #321 [???]
• Other topics
  •  

15:55 → 16:00 AOB