Rucio Meeting

Thursday 23 Sept 2021, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

1 News

• Rucio workshop commencing soon!

2 Community News & DevOps roundtable

• Fermilab
  • Working on upgrading DUNE database
  • Vera Rubin / Hermes discussion to automatically ingest data into Data Butler
    • RabbitMQ streams - have a look
  • Multi-VO discussion @ workshop
• Belle II
• LDMX
  • waiting for 1.26.5
• ATLAS
  • Data Challenge - need to review
• CMS
  • upgrade to 1.26 (Tuesday) - so far so good :-)
  • move to flux
  • connected all sources of popularity to trace generation
    • WMS (prod + analysis), physics apps, xrd read/write instrumentation
    • missing: running bare root at their local installations
• STFC
  • development work on policy packages - generalising it w.r.t. multi-vo
  • customise module loading on client side
  • PR: elimination of config file
    • needs ticket (or update of existing ticket)
• RAL
  • 1.26.2 running
  • setting up Web-UI
• ESCAPE
  • waiting for 1.26.5 (incl. web-UI)
• WLCG
  • yesterday mock data challenge for ATLAS (600k, 10TB)
  • today mock data challenge for CMS (100k, 1TB)
  • some issues with monitoring (e.g. avg throughput calculation)
  • check expected vs actual nr_bytes moved

3 Hot topics

•  

4 Developers roundtable

Rucio 1.27 "Batdonkey v. Superdonkey" priority followup

• In Progress
  • Auditor overhaul #3437 [Dimitrios]
    • Will discuss with Dimitrios + CMS + Martin once they are back
  • rucio.cfg vs config table #2630 [Mario, David]
    • Needs review
  • Logging review #4220 [Martin, All comp leads]
    • switch from logging to logger - still a few daemons left
  • Disentangle fts3 specific code from conveyor and move to transfertool #857 [Radu]
    • Work almost done
    • One open PR for submitter-stager merge
    • One other PR already prepared, will submit once previous merged
    • Poller ongoing
    • Lots of changes incoming
  • helm-charts release management #4794 [Radu, Eric, Martin]
    • Eric submitted comments to the issue
    • Switch to the new naming scheme first, thus next release would be 1.26.0
    • Needs to be described on the documentation
    • Github action needs to be upgraded to look into separate release branches
      • Probably not a big change
  • Quality of Service #3419 [Matt]
    • Demonstrator for sotrage-issued QoS changes for BNL MAS
    • Matt should have dev changes done in the coming weeks
• Todo
  • Get SSO Login working [Rizart]
    • no news
  • Down-scoped tokens for user interactions #4791 []
  • Optimize database interactions #4793 [Martin, Mario, Radu]
  • Prepare replacement of current policy import with policy packages #4798 [James, Martin]
    • cf. news
  • Rename Daemons to commonly understandable names #4795 [Martin]
  • Versioning for REST API #4796 [Ben, Martin]
  • Enabling tests for different policy package #3878 [Mayank]
• Done
• Delayed

Developer roundtable

• Docker auto builds [Mayank]
  • Will not use docker auto builds anymore, instead have GH actions build and push the containers
  • Workflow PR is in place
  • Secrets for Docker are in place
  • Right now builds daemons, clients, server; will add the other auto-builds as well
• Enabling http on xrootd containers
  • Ilija might have managed to do that (plugins? config? scitokens?)
• Changing CI oracle-express database from 11 to 18
  • PR done, merge asap

5 AOB

•