Rucio Development Meeting

Thursday 15 Jul 2021, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• Rucio 1.26 progress
  • RC2 just released, testing going very well
  • Final in the next days
• Rucio Special Interest Groups
  • Activity to bring together people with a special interest in a specific topic (involving) Rucio
  • Inform and learn from each others plans, tests, outcomes; plan developments;
  • Discussions/coordination via dedicated mailing lists / chats
    • Dedicated meetings only if needed, otherwise discussion within the Rucio Dev meeting
  • Current (just started) SIGs
    • Metadata
      • rucio-sig-metadata@cern.ch
      • Metadata evolution in Rucio, community usecases, benchmarks, etc.
    • QoS
      • rucio-sig-qos@cern.ch
      • Storage QoS in Rucio
      • We will start from prototyping a simple "storage-self managed QoS" usecase and evolve the system from there
  • Possible SIGs
    • DIRAC
      • Might be useful to coordinate the different efforts from Rucio for a DIRAC interface
      • Opinions?
    • ?
• Rucio 1.27 "Batdonkey v Superdonkey" release planning in Dev meeting on Aug 12

15:05 → 15:15 Community News & DevOps roundtable

• ATLAS
  • Deployed 1.26
    • rse_id index on replicas
      • Caused big load on database due to non-optima query plans
      • Set the index to invisible
      • For anyone running oracle:
        • Monitor the deployment and look at query plans
      • Index is not used at the moment anyway
• CMS
  • Some Transfers stay in QUEUED state forever
    • Possibly fixed with 1.26?
    • Lack of logging messages, very hard to understand what is happening
      • Radu: Planning to add more information
• Fermilab
  • Delegation of proxies to FTS3
    • Extend cron to include credentials
  • Rubin Obervatory discussions on how to integrate Rucio in their data framework
• Belle II
  • Upgrade to 1.26.1 when it is out
    • .1 because still waiting for some PRs to be merged
• SKAO
  • Metadata
    • SKAO working on metadata PR
    • Evolving equality engine PR from Gabriele
  • Deployment
    • SSL Ingress issues
    • Keeping up with helm-charts and changes there
• DUNE/Edinburgh
  • LFN2PFN algorithm for DUNE metadata catalog
    • One code change missing, should not be controversial (for 1.26.1?)
  • Light-weight client plan
    • Make client more flexible in terms of locally installed tools
      • GSOC student also working on this
• ORNL
  • Looking into Rucio
• ESCAPE
  • Fully configured OIDC rucio instance
    • Users already testing, works well
  • After summer commencing full-stack tests
  • Tests with transmogrifier
    • #4728

15:15 → 15:25 Hot topics

15:25 → 15:55 Developers roundtable

• Big backlog of PR
  • Big backlog of PRs due to Code Freeze (Should be over on Monday)
  •  
• Datasetlocks not getting properly deleted #1860
• Issue with lost source
  • Possible due to race-condition when rules gets STUCK
    • sources are cleared out
    • Only new sources created when transfer is re-submitted, but might take time
    • In this time-window, the replica might be unprotected
    • Possible solutions:
      • Check if there is a REPLICATING replica, and not delete them
      • Keep requests (and sources)?
  • Ticket #

15:55 → 16:00 AOB

• Next week no meeting (Summer schedule)!