Rucio Meeting

Thursday 16 Sept 2021, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

1 News

• Hacktoberfest 2022 coming up
  • https://hacktoberfest.digitalocean.com/
  • Sign up 1-31 October, 4 PRs, first 55k who complete get a shirt
•  

2 Community News & DevOps roundtable

• ATLAS
  • Release 1.26.4 regression affecting undertaker (<-- broken)
    • hotpatched
    • no hotfix release, move directly to 1.26.5
• CMS
  • CTA recall
    • multi-hop jobs queued on CTA side without FTS job?
    • retrial strategy?
      • commit exists, not merged yet
      • needs other PRs merged first
  • Hi Joao
    • destination file already exists #4447
    • FTS prepping to release next week
      • fts get_metadata -> string to json
      • archive_monitoring, destination_file_report
        • if file exists then add destination report json to get_metadata
      • Radu volunteered :-)
      • conveyor-receiver will miss not miss it, it's in transfer-complete json
        • but not in ES
• Fermilab
  • Icarus 1.26.2, DUNE next 1.22 (with custom schema patches :-)
  • Rubin team to test Rucio
• Belle II
  • 1.26.4 with patch
• LDMX
  • waiting for 1.26.5 fix :-)
• ESCAPE
  • deleted_dids not being filled
    • requires a trigger(?)
    • Cedric will followup
  • Datalake as a service is complete
  • WLCG
    • ATLAS scratchdisk tests successful
    • Mock-run on datadisk before actual challenge
    • purge_replica functionality available at rule creation?
      • not in CLI
      • showstopper for CMS due to greedy mode / policy (?)
• SKAO
  • process of moving to 1.26
• RAL
  • process of moving to 1.26 (Monday)

3 Metadata tests in Belle II

Speaker: Cedric Serfon (Brookhaven National Laboratory (US))

2021-09-16 - Metadata tests in Belle II.pdf

4 GSOC: New protocols for exascale data management with Rucio

Speaker: Rakshita Varadarajan

Google Doc

5 Developers roundtable

Rucio 1.27 "Batdonkey v. Superdonkey" priority followup

• In Progress
  • Auditor overhaul #3437 [Dimitrios]
    • Will discuss with Dimitrios once he is back
  • rucio.cfg vs config table #2630 [Mario, David]
    • Needs review
  • Logging review #4220 [Martin, All comp leads]
    • No news
  • Disentangle fts3 specific code from conveyor and move to transfertool #857 [Radu]
    • Work ongoing
    • One open PR for submitter-stager merge
    • One other PR already prepared, will submit once previous merged
  • helm-charts release management #4794 [Radu, Eric, Martin]
    • Eric submitted comments to the issue
    • Switch to the new naming scheme first, thus next release would be 1.26.0
    • Needs to be described on the documentation
    • Github action needs to be upgraded to look into separate release branches
      • Probably not a big change
  • Quality of Service #3419 [Matt]
    • Demonstrator for sotrage-issued QoS changes for BNL MAS
    • Matt should have dev changes done in the coming weeks
• Todo
  • Get SSO Login working [Rizart]
  • Down-scoped tokens for user interactions #4791 []
  • Optimize database interactions #4793 [Martin, Mario, Radu]
  • Prepare replacement of current policy import with policy packages #4798 [James, Martin]
  • Rename Daemons to commonly understandable names #4795 [Martin]
  • Versioning for REST API #4796 [Ben, Martin]
  • Enabling tests for different policy package #3878 [Mayank]
• Done
• Delayed

Developer roundtable

• Docker auto builds [Mayank]
  • Will not use docker auto builds anymore, instead have GH actions build and push the containers
  • Workflow PR is in place
  • Secrets for Docker are in place
  • Right now builds daemons, clients, server; will add the other auto-builds as well
• Enabling http on xrood containers
• Changing CI oracle-express database from 11 to 18
  • PR done, merge asap

6 AOB

•