Rucio Meeting

Thursday 11 Apr 2024, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• CHEP Abstract submissions

15:05 → 15:15 Community News & DevOps roundtable

• ATLAS
  • RSE Availability request from ATLAS
    • Right now binary, e.g. WRITE on or off
    • Request to make this more flexible WRITE on, off, testing (just activate rse for write testing)
    • Interest from other communities?
      • CMS would be interested in it, they have a manual solution for this
• CMS
  • Upgrade to 34 planned
  • replicas_history
    • Not being implemented in Rucio (yet), need to re-visit if it is useful functionality
    • Usecase: Check if a did was ever available
  • Double counting of multiple rules (of the same user) touching the same replica
    • AccountCounter
• Fermilab
  • RUBIN & all Fermilab Rucios running 34
• PIC
  • Switch to Tokens (on Rucio 32 LTS)
    • Installed Indigo IAM (did not work out of the box with keycloak)
• CERN IT
  • Preparing prototype intances for SMEs
• RAL
  • Moving to Rucio 34

15:15 → 15:45 Rucio-SENSE interoperation prototype

Speaker: Aashay Arora (Univ. of California San Diego (US))

DMM.pdf

15:45 → 15:55 Developers roundtable

Progress-2024-04-11.png

• Rucio 35 "Donkey and the Data Factory" roadmap
  • In Progress
    • Review operations where SSL verification is disabled #6632 [Dimitrios]
      • Split between RHEL CA which are trusted and our own grid-cas (/etc/grid-security/)
      • Support of letsencrypt?
      • Container start
        • If additional certs are in /etc/grid-security (usually not the case)
        • On container start they are added to system trust 
    • Collect functional RSE attributes into a single file #6624 [Dimitrios, Mira]
    • Remove all instances of mutable objects (e.g. empty lists or dictionaries) as default parameter values #5169 [Erling]
    • Finalize SQLAlchemy 1.0 -> 2.0 syntax migration #6640 [Erling]
    • Prepare replacement of current policy import with policy packages #4798 [James]
    • Webui 35 tracker 1/34 [Mayank]
    • Formatting / style guide #287 [Riccardo, Dimitrios, Maggie]
    • Type annotate the code #6588 [Riccardo]
  • In Review
    • JupyterLab version (v3 to v4) update #27 [Francesc, Enrique]

• • Todo
    • Unable to Delete File DID via Undertaker #5154 [Alex]
    • factorize duplicate messaging code into a common module or class #6423 [Alex]
    • SUBSCRIPTIONS_HISTORY integrity error #6292 [Cedric]
    • Make new token support Multi-VO compatible #6406 [Dimitrios]
    • Use OpenID Connect Discovery #6414 [Dimitrios]
    • Allow for token providers other than IAM #6630 [Dimitrios]
    • Make WebDAV protocols on par with GFAL #6633 [Dimitrios]
    • bridge the gap between running rucio in demo env and full production deployment #187 [Enrique, Hugo]
    • Retire RSE limits #6629 [Erling, Dimitrios]
    • Rename internal API layer to gateway #6644 [Erling]
    • Prevent multiple Reaper threads from working on the same replicas #6512 [Hugo]
    • Make a table for all daemon and its functionality #296 [Hugo]
    • Re-write and standardization of client command structure #6639 [Maggie]
    • Evaluate all TODO and FIXME comments #6642 [Martin]
    • Testing: Client Only tests #6599 [Mayank, Maggie]
    • Evaluate archive workflows for rules (Using an archive as a source) #6643 [Riccardo]
    • Improving docstrings #6628 [Riccardo]
    • Create and populate common monitoring repository #6641 [Tim]
  • Done
  • Delayed
• Documentation corner
  • Documentation and dev guidelines for Mypy type annotations #116 [Mayank, Martin]
  • Document environmental variables affecting the client #171 [Dimitrios]
  • Improve documentation on rucio.cfg vs configuration table #183 [Maggie]
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Add instruction about DB partitioning #185 [Martin]
  • bridge the gap between running rucio in demo env and full production deployment #187 [Enrique, Hugo]
  • Introduce documentation on subscriptions #190 [Cedric]
  • Add Rucio QoS RSE description and instructions #268 [Matt]
  • Document how to set up command line argument completion #275 [Bouwe]
  • Formatting / style guide #287 [Riccardo, Dimitrios, Maggie]
  • Document how deletion occurs. #288 [Anil]
  • Instructions for tape collocation plugins #289 [Maggie]
  • Documentation for RSE settings #293 [???]
  • Make a table for all daemon and its functionality #296 [Hugo]
  • Testing instructions #309 [Maggie]
• Other topics
  •  

15:55 → 16:00 AOB