Rucio Development Meeting

Thursday 18 Feb 2021, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:10 News

• Two releases this week
  • 1.24.3.post1
  • 1.24.4
• Release schedule
  • 1.24.5 Feb-22
  • 1.23.13 LTS Feb-22
  • 1.20.13 LTS Feb-22
  • Code Freeze from Feb-24
    • 1.25.0rc1 Feb-25
    • 1.25.0 Mar-1 ... Mar-5

15:10 → 15:20 Community News & DevOps roundtable

• RAL/MultiVO
  • No news
• ATLAS
  • No news
• CMS
  • No news
• Belle II
  • No news
• DUNE/Edinburgh
  • No news
• ESCAPE
  • CNAF-STORM deletion worked
  • Only way to delete was not via reaper2 --greedy but with specific RSE greedyDeletion=True attributes
    • This is a bug in Rucio, will be fixed
  • First integration of HPC in the datalake
• FNAL/DUNE
  • DUNE running Rucio deployment on OpenShift in production now
    • NOTFOUND exceptions in reaper2 
      • But gfal-stat shows that the file exists
      • Possibly a certificate issue?
      • Possible related fix from CMS in gfal

15:20 → 15:30 Hot topics

15:30 → 15:55 Developers roundtable

• Rucio 1.25 "Rat-Donkey" release followup
  • In Progress
    • Stronger integration of Globus Online transfertool #4216 [Matt, Ben]
      • PR is ready, Martin need to review
      • Need to add documentation to describe how to run GO with Rucio
    • Deprecate reaper1 #4213 [Martin]
      • Submit this afternoon
    • Logging review #4220 [Component Leads]
      • Internal deadline: Feb-11 (passed)
      • still ongoing
      • New loggers all added
        • Setup is there, some daemons are missing the prefix (heartbeat)
      • PLEASE go through your components and cleanup log messages
    • Versioned (History) Tables should be defined explicitly #2063 [Martin]
      • Should be in 1.25
    • Client ticket cleanup [Mario, Radu]
  • Todo
    • Test new rule mode and switch it to default #4215 [Martin]
      • ESCAPE already testing this quite successfully
      • Testing it in ATLAS - activate next week
        • Will be needed, otherwise won't run on K8s
    • Migrate documentation to new docusaurus [Martin]
      • Bring in sync readthedocs docu to new rucio/documentation, Daniel will help
      • API documentation generation does not work
  • Done
  • Delayed
    • Quality of Service #3419 [Mario, Martin, Rizart, *]
    • rucio.cfg vs config table #2630 [Mario]
    • Remove webpy endpoints and dependency #4044 [Ben]
      • FLASK endpoints running on ATLAS integration
        • Needs more testing (Authentication, Traces, WebUI)
      • Will leave web.py in 1.25 --> aimed to 1.26
    • Identify and cleanup unused functionality and code #4221 [Component Leads]
• Discussion: Getting rid of oracle triggers?
  • Martin is in favor
  • Mario is in favor
  • Check with Oracle people at CERN about triggers
    • If nothing speaks against it, remove them with 1.25/1.26
• Full stack upload/download/transfer testing [Mayank]
  • xrootd protocol upload/download tests PR exists (Needs review!)
    • Ready to merge; Change with uploadclient can be submitted later
  • TPC PR
    • Should be ready soon for review
• Parallel testing [Ben]
• In memory database [Mario]
  • Possibly crashing due to memory limitation?
• Documentation [Daniel, Martin]
  • Move k8s tutorial into main doc, need to sort out where exactly
  • Extending docs with more details
  • Add "What is Rucio tab" to sidebar on webpage?
    • Martin or Mario can write something
  • Add new documentation content already to new rucio/documentation
  • Move k8s-demo to rucio/ repository
    • Martin will create rucio/rucio-k8s-tutorial
    • Thomas will push repo there
•  

15:55 → 16:00 AOB

• GO discussion in DOMA General meeting next week