Rucio Meeting

Thursday 24 Mar 2022, 15:00 → 16:00 Europe/Zurich

Zoom only

Martin Barisits (CERN)

15:00 → 15:05 News

• 1.28 progress
  • 1.28.0rc1 validation uncovered a deadlock issue with transfers
    • 1.28.0rc2 and 1.27.11 will be released today
  • 1.28.0 Final on Monday
    • Code Freeze lifted then
• 1.29 release planning next week!
• WebUI survey
  • Please participate in the survey!
  • https://docs.google.com/forms/d/e/1FAIpQLSdpD81jcjC4CScK8O_gJdfSw3ze4uvuKuEROrOaN_V74LmXCg/viewform

15:05 → 15:20 Community News & DevOps roundtable

• ATLAS
  • Tape challenge
  • #5374 dark reaper issue
    • Heartbeat never refreshed in the dark reaper -> does not work with multiple threads
    • Reduced to 1 thread for the moment
    • Already a PR to fix this
      • Will see in which release this goes 1.28.0 or 1.28.1
  • Deadlock issue with 1.28.0rc1 + 1.27.10
    • Main issue was a forgotten index
      • Caused a deadlock issue on database side
      • Solved by adding index
    • Secondary race condition in the finisher
      • Multiple finisher could work on the same transfer path in a MH transfer
      • Also fixed with 1.28.0rc2 + 1.27.11
  • Testing of  temporary tables for reaper (Activated via config variable)
    • Replacement of complex OR-chunked queries
    • ATLAS activated this globally on all RSEs, works well
      • No negative (nor positive) performance impact
      • But queries and workflows are much simpler
  • Flux2
    • Motivated by the recent CERN kubernetes incident and handling of secrets
    • Secrets are now kept (encrypted) in the repository
      • Much easier if all clusters are wiped and secrets are lost
• CMS
  • Running probes based on 1.27 release
• Fermilab/DUNE/RUBIN/...
  • Move from OpenShift to raw Kubernetes
    • All Fermilab clusters on k8s primitives now
      • Will mean better support throughout the board
• Belle II
  • DIRAC Python 3 move
    • In preparation for Rucio 1.29.0 which drops Python 2.7
  • Policy package for Belle II
  • On 1.26.X at the moment, so can't test temp tables yet
• ESCAPE
  • Closing workshop for ESCAPE last days
    • Rucio was very well received and are either moving to Rucio or evaluating
    • Rucio Jupyter plugin + Data Lake as a Service was mentioned many times!
  • ESCAPE will continue to exist as a "forum" (Collaboration agreements being prepared)
• SKAO
•  

15:20 → 15:40 1.28 release retrospective

Speaker: Martin Barisits (CERN)

Rucio 1.28 retrospective.pdf

15:40 → 15:55 Developers roundtable

Progress-2022-03-24.png

•  

15:55 → 16:00 AOB