Rucio Meeting

Thursday 3 Nov 2022, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• Rucio Workshop next week!
  • Very exciting and dense schedule
  • No Rucio meeting next week
• November release schedule
  • 1.29.8 Nov-02
  • 1.29.9 Nov-14
  • 1.26.15 LTS Nov-14
  • 1.29.10 Nov-21
  • 1.30 code freeze Nov-23
  • 1.30.0rc1 Nov-25
  • 1.30.0 Nov-30
• Release names 2023 - Thanks for voting!
  • Rucio 1.31 “Donkeys of the Caribbean”
  • Rucio 1.32 LTS “The Good, The Bad and the Donkey”
  • Rucio 1.33 “Eternal Sunshine of the Donkey’s Mind”
     

15:05 → 15:20 Community News & DevOps roundtable

• ATLAS
  • Hermes activemq resilience
    • ActiveMQ brokers changed frequently lately, hermes not good in picking this up
• CMS
• Fermilab / DUNE / ICARUS / ...
  • Production OKD cluster coming up
    • Moving services over there
    • Should allow to upgrade DUNE and ICARUS to 1.29
  • RUBIN
    • Production and Devel cluster running on K8s
• Belle II
• ESCAPE
  • Configure S3 endpoint
    • FTS works fine
    • Rucio does not
  • Conveyor-poller
    • Issue with OIDC tokens
• RAL / MultiVO
• SKAO
  • Trying hierarchical data movement
•  

15:20 → 15:25 Hot topics

15:25 → 15:55 Developers roundtable

Progress-2022-11-03.png

• Rucio 1.30 "The Donkeynator" progress tracking
  • Release date will be end of November
    • Code freeze on Nov-23
  • In Progress​​
    • Create a contributor guide for probes #84 [Dimitrios]
    • Exchange `Tombstone` function-based indices with normal indices #5440 [Mario, Martin, Radu]
    • Unable to Delete File DID via Undertaker #5154 [Martin]
      • Does not work on 1.26
      • Need unit test on 1.29
    • Rules on containers in state OK but not all the files from the containers have locks #5447 [Martin]
    • Add python 3.10 tests to CI framework. #5145 [Mayank]
      • Tests with sqllite fail to bootstrap on stream8/9 because of timeout
    • Release webUI 2.0 beta version [Mayank]
      • Final PR from Nimish (GSOC) there
      • Next step:
        • helm-chart to deploy on ATLAS-integration
        • Testing and fine-tuning
    • Switch to pytest [Radu]
      • 2/3 of test files finished and merged
      • 1/3 to be done soon
    • Re-work/cleanup distance table #5454 [Radu]
  • In Review
    • Adherence to rule grouping dependent on order of ATTACH #5251 [Igor]
    • rework prometheus/statsd metric names #5804 [Radu]
  • Done
    • Further ingetragtion of CMS-style consistency checking #5438 [Igor]
    • Remove old readthedocs documentation once the new one is fully operational #5446 [Joel, Martin]
    • Introduce heartbeat handler to all daemons #5443 [Cedric]
    • rework conveyor-throttler #5805 [Radu]
    • Create documentation page for S3-type storage #5450 [Rizart]
  • Todo
    • Create a server/daemon installation howto #5445 [Joel, Mayank]
      • To discuss with Martin, Mayank, Anton
    • Create developers testing guide in the documentation #5452 [Joel, Mayank]
    • foreign key error on deleting dids in reaper #5733 [Martin]
    • Merge list_dids and list_dids_extended methods #5448 [Rob]
  • Delayed
    • Refactor the different test workflows to a common one [Ben, Mayank]
    • Add symlinks for the new daemon names #5131 [Joel]
    • Prepare a clear list of changes which need to be made to use new daemon names #5132 [Joel]
    • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
    • Reduce rule tickets to les than 20 [Martin]
    • Reduce logging of server & daemons #5449 [Mayank
    • Add support for S3 virtual hosted buckets #5451 [Rizart]
      • Test on-premise
      • Ongoing support issue with AWS/Fresno X.509 shenanigans
• Other topics
  •  

15:55 → 16:00 AOB

• Cedric gives a Database Overview talk at the Rucio workshop next week
  • Will reach out to people to get some stats, comments, etc.