Rucio Meeting

Thursday 11 Nov 2021, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

1 News

• Release schedule
  • 1.26.9 just released
    • Last regular release in this release line
    • Supported until July 2023 with LTS patch releases
  • 1.27.0
    • Code freeze Nov 17!
      • Some reviews missing, please be ready to address them
    • RC1 shortly after
    • Final (hopefully) still this month
• Release schedule for 2022
  • Need release name suggestions!
• Next meeting
  • Small talk from Radu about new container/helm workflow for rucio.cfg

2 Community News & DevOps roundtable

• ATLAS
  • Nothing to report
• CMS
• Fermilab
• DUNE
  • Client-side policy packages development
• DIRAC
  • Converting DIRAC-Belle2 Rucio integration to MultiVO-DIRAC
    • Issue with `/` in container/dataset name ignored on Server
      • Is part of the schema, but it somehow gets filtered within Rucio
    • Works for Belle 2 - Could be difference between Single-VO and Multi-VO deployment?
      • Issue: #
  • Permission needs to be addressed as well after the above issue is solved
  • DIRAC interface for now only addresses upload
    • Probably no other methods needed, but not entirely clear yet
• Multi-VO
• SKAO
  • Moving from 1.23 to 1.26
    • Some issue with the salting (Which changed in 1.24)
• ESCAPE (In Absence)
  • Some issues with reaper greedy deletion
    • Not really fast enough, might be an issue due to a sign_url crash in the loop?
  • Hermes uses large amount of memory for ESCAPE
    • needs investigation

3 Hot topics

4 Rucio-SENSE Interoperation

Speaker: Jonathan Guiang (Univ. of California San Diego (US))

rucio-sense_11-11-2021.pdf

5 Developers roundtable

Rucio 1.27 "Batdonkey v. Superdonkey" priority followup

• In Progress
  • Auditor overhaul #3437 [Dimitrios, Eric, Stefan] [Longer activity lasting beyond 1.27 release]
    • Let's schedule meeting in November
  • Logging review #4220 [Martin, Joel, All comp leads]
    • Would be good to get into 1.27
  • Quality of Service #3419 [Matt] [Beyond 1.27 release]
    • Demonstrator for storage-issued QoS changes for BNL MAS
    • DOMA QoS
  • Optimize database interactions #4793 [Martin, Mario, Radu] [Beyond 1.27 release]
    • Radu started to look into temp tables
      • Proof of concept works, but different :-) on Oracle
        • Global vs private temporary tables
          • Global needs to be part of schema
    • Meeting with CERN IT DB admins
      • Decrease Transaction (7000 transactions/s)
        • Recently found issue about session handing between API and Core
        • Will be addressed to largely reduce the number of empty rolled back sessions
      • Decrease LOGON rate (7-10 logons/s)
        • Optimization session pools
      • Temporary tables
  • Rename Daemons to commonly understandable names #4795 [Martin, Joel]
    • Add aliases now, remove them later (if at all)
    • If you have suggestions for proper names, please add them to the issue
      • Deadline for this next week
  • Prepare replacement of current policy import with policy packages #4798 [James, Martin] [Beyond 1.27 release]
    • Best method to get policy packages into container?
      • Probably not a good idea to pip-install them on startup
    • Fermilab + CMS builds separate containers including these
  • Enabling tests for different policy package #3878 [Mayank]
    • New GH action workflow
      • Tests are specified via build-matrix 
        • Specify modifications of rucio.cfg
        • Right now it assumes policy package is in container
• Todo
  • Get SSO Login working [Rizart]
  • Down-scoped tokens for user interactions #4791 [] [Beyond 1.27 release]
  • Versioning for REST API #4796 [Ben, Martin] [Beyond 1.27 release]
• Done
  • Disentangle fts3 specific code from conveyor and move to transfertool #857 [Radu]
  • helm-charts release management #4794 [Radu, Eric, Martin]
  • rucio.cfg vs config table #2630 [Mario, David]
• Delayed

Developer roundtable

• Metadata
  • PR needs merging, Review is GTG

6 AOB