Rucio Meeting

Thursday 4 May 2023, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• May meeting schedule
  • May 4 (today)
  • May 11 CANCELLED (CHEP)
  • May 18 CANCELLED (CERN Holiday)
  • May 25
• May release schedule
  • May 3 1.26.17 LTS
  • May 3 1.31.2
  • May 15 1.31.3
  • May 29 1.31.4
• Testing of temporary tables
  • If you run 1.28+ please consider testing Rucio with temporary tables
  • [core]temp_tables=True
  • We want to make this the default with Rucio 32 and remove the conventional workflows completely

15:05 → 15:25 Community News & DevOps roundtable

• ATLAS
  • "Trying" to test Rucio GLOBUS in production
• CMS
• Fermilab DUNE / ICARUS / RUBIN
  • ICARUS
    • RSE setup @ SLAC
    • Setting up a Hypot
  • RUBIN
    • Issue using RALs FTS to execute transfers
      • Large rules being submitted to FTS in bulk
      • At some point switch to single submission instead of bulk
        • Only happens at RAL FTS
        • Links configured specifically so only single source can be used
      • Start checking logs
        • Not MultiSource jobs
• DUNE
  • Migrated metadata to metacat
    • Big milestone for DUNE
• Belle II
  • Problem with deletions
    • Not with Rucio itself, but the database
    • Big deletion campaign ongoing, increased #threads (default 2, during campaign 4 causes issues)in reaper, caused some issues on DB side
      • sync between primary and secondary DB taking longer causing memory issues on primary
• ESCAPE
  • CMS setup script for environment including gfal2
    • Eric will send script
• RAL / MultiVO
  • Token Auth not working (via CLI)
    • See #support

15:25 → 15:45 Python 3.6 deprecation discussion

Python 3.6 deprecation.pdf

• All present communities agreeing to move ahead with the proposal
  • Since older clients are forward compatible, 1.29 or 1.31 can be used

15:45 → 15:55 Developers roundtable

Progress-2023-05-04.png

• 32 "The Good, The Bad and the Donkey" Priority followup
  • In Progress
    • Migrate DID and DID-metadata view to clean architecture #w/160 [Mayank, Anton]
    • Migrate List My Rules and Rule View to clean architecture #w/159 [Mayank, Anton]
    • Create Pages for Create Rule Workflow #w/144 [Anton]
    • Adapt probes to rucio/rucio#5804 #p/108 [Eric]
    • Internal transfer monitoring #6189 [Radu]
    • New dispatcher logic for conveyors #6190 [Radu]
  • In Review
    • Unable to Delete File DID via Undertaker #5154 [Anton]
  • Todo
    • Design and implement a token request module #6187 [Dimitrios]
    • Incorporate IAM-based testing for token workflows #6188 [Dimitrios]
    • Design and implement a token cache module #6183 [Dimitrios]
    • Support protocols with NULL priority #6201 [Dimitrios]
    • Kick off SIG-Tokens [Martin]
    • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
    • Reduce core tickets <15 [Martin, Mario]
    • foreign key error on deleting dids in reaper #5733 [Martin]
    • Create a server/daemon installation howto #d/178 [Mayank]
    • Create developers testing guide in the documentation #d/177 [Mayank]
    • Support OIDC login #w/149 [Mayank]
    • Migrate Dashboard to Clean Architecture #w/158 [Mayank, Anton]
    • Exchange of function-based indices with normal index #6143 [Radu]
    • Update/Re-design core.meta module #5224 [Rob]
  • Done
    • Replace all instances of X509 identities presented in the legacy format #6182 [Dimitrios]
      • One PR still open for the documentation
  • Delayed
• Documentation corner
  • Document environmental variables affecting the client #171 [Dimitrios]
  • Documentation and dev guidelines for MyPi type annotations #116 [Mayank, Martin]
  • Create developers testing guide in the documentation #177 [Mayank]
  • Create a server/daemon installation howto #178 [Mayank]
    • Ubuntu-specific PR #215
  • Improve documentation on rucio.cfg vs configuration table #183 [Radu]
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Add instruction about DB partitioning #185 [Martin]
  • bridge the gap between running rucio in demo env and full production deployment #187 [Radu]
  • Introduce documentation on subscriptions #190 [Cedric]
  • improve documentation about --distance and --ranking and source ranking #199 [Radu]
• Other topics
  • CMS consistency code & documentation in new repository rucio/consistency-enforcement
  • Versioning of documentation?
    • Not at the moment
    • Maybe mentioned since when a certain feature is availabile (Rucio version)

15:55 → 16:00 AOB