Rucio Meeting

Thursday 31 Oct 2024, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• November meeting schedule
  • Nov-07
  • Nov-14
  • Nov-21
  • Nov-28

15:05 → 15:25 Community News & DevOps roundtable

• ATLAS
  • Proton run!
  • Tape Metadata turned on last week, works well
    • colocation hints as well as activity priority
• CMS
• Fermilab DUNE / RUBIN / co
  • Updating policy packages for all deployments
    • Changes of type annotations in permissions
  • DUNE
    • Expired files handling
      • What DIDs get preserved (when is it put to deleted_dids) -> ticket#
      • When is a message generated to notify about the removal -> ticket#
      • We need to document this -> Ticket for the docusprint!
        • ERASE not part of the message documentation
        • All configuration options should also be collected
      • Open PR for deletion doc #295
    • Debugging for files not transferred
      • rule suspension, state machine not really explained
• CERN IT
• Edinburgh DUNE
  • CiLogon token testing
    • Basic usage is working, but fts transfers will require changesin CiLogon
• ALTO

15:25 → 15:55 Developers roundtable

Progress-2024-10-31.png

• 36 "Donkey Unchained" roadmap
  • Todo
    • Allow for token providers other than IAM #6630 [Dimitrios]
      • Depend on #6406
    • Implement new token authentication for download #7029 [Dimitrios]
      • Depend on #6638, but is also top priority
    • Prevent multiple Reaper threads from working on the same replicas #6512[Hugo]
    • Evaluate archive workflows for rules (Using an archive as a source) #6643[Riccardo]
  • In Progress
    • factorize duplicate messaging code into a common module or class #6423 [Alex]
    • Make new token support Multi-VO compatible #6406 [Dimitrios]
    • Create a new API endpoint for the clients to request tokens #6638 [Dimitrios]
      • Will be experimental, but is top-priority
    • Review operations where SSL verification is disabled #6632 [Dimitrios]
      • One part will be done, another not (client parts)
    • Retire RSE limits #6629 [Erling, Dimitrios]
      • Martin-Check
    • Prepare replacement of current policy import with policy packages #4798 [James, Martin]
      • Belle II policy can be removed
      • ATLAS policy DONE
    • WebUI Tracker 36 [Mayank, Viktoriaa]
      • X509 working on all browsers
        • Will require a bit of change on Rucio server
      • did-schema with special characters
        • New config options for the encoding
    • Refactor all ATLAS-related code into ATLAS policy package #7027 [Riccardo]
    • Better define unit tests and integ tests #7007 [Riccardo]
    • Improve test coverage #7008 [Riccardo]
  • In Review
    • Re-write and standardization of client command structure #6639 [Maggie]
      • Please review! (Need comments from Dimitrios, Mario, Riccardo)
        • Dimitrios will look on it tomorrow (Oct-25) - design/interface from user PoV
      • Needs a rebase for the codecov spam
      • We should merge Erics rich-clients PR first, otherwise rebase will be difficult since Erics internship ended
  • Done
    • Allow permissions to optionally return a message #6580 [James]
  • Delayed
    • Make WebDAV protocols on par with GFAL #6633 [Dimitrios]
    • Improve docstrings #6628 [Riccardo]
• Documentation corner
  • Document activities used by rucio-daemons #326 [???]
  • Generate "Configuration parameters" documentation page automatically, to avoid mismatches between documentation and code #325 [???]
  • Error when trying to upload file with custom name containing slashes #336 [???]
  • Make a table for all daemon and its functionality #296 [Hugo]
  • Documentation (developer): Add guidelines for testing API-related changes #322 [???]
  • Documentation for RSE settings #293 [???]
  • Document how deletion occurs. #288 [Anil]
    • PR #295 underway
  • Introduce documentation on subscriptions #190 [Cedric]
  • Add instruction about DB partitioning #185 [Martin]
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Changing auth_type should invalidate current Rucio authn token #321 [???]
• Other topics
  • Riccardo: discussion on opt-in vs opt-out for archive metadata for RSE (e.g. having enable_rse_attribute vs disable_rse_attribute as RSE attribute)
    • When should the tape metadata generation be used
      • Archive metadata policy package needs to be enabled and Destination RSE needs to be rse_type.TAPE
      • Add an attribute to enable/disable the generation for an RSE?
      • For now, since there is not a strong need for a per-tape switch, send it for everything (if a policy is defined)
        • However, if we see issues with specific tape RSEs, we might add a per-RSE switch later

15:55 → 16:00 AOB