Rucio Development Meeting

Thursday 9 Jan 2020, 15:00 → 16:00 Europe/Zurich

4/S-030 (CERN)

Martin Barisits (CERN)

1 News

• 3rd Rucio Community Workshop
  • March 10-12 2020, Fermilab US
  • March 09 2020 will be a DOMA Face to Face meeting at Fermilab
  • March 12-13 possibly a Rucio Coding Camp at Fermilab, still to be finalized
• CERN-HSF Google Summer of Code
  • Please collect ideas, will discuss next week
  • To be submitted until Feb 04
• Development meeting
  • New room
  • Now 60 minutes (instead of 90!)
• Development meetings January & February 2020
  • Jan 09
  • Jan 16
  • Jan 23
  • Jan 30
  • Feb 06
  • Feb 13 ATLAS S&C week
  • Feb 20
  • Feb 27
• Release schedule
  • Jan 20 - 1.21.5
  • Jan 24 - 1.21.6 [Special release only including sharding simplifications]
  • Feb 03 - 1.21.7
  • Feb 14 - 1.21.8
  • Feb 24 - 1.21.9
  • Mar 02 - 1.22.0rc1

2 News from the experiments

• ATLAS
  • Interest of stopping the old reaper
    • Issue with tombstoned COPYING replicas
    • Seems to grow substantially in memory over time
      • Cedric will have a look
  • Otherwise running quite smoothly over Christmas
• CMS
  • Usage and future of the Probes
    • Currently used within Nagios, future with K8s?
    • Possible solution for K8s: Jobber
      • Eric will take a look
• ESCAPE
  • Nothing to report
• Multi-VO
  • Eli will return to the project at the end of the month
  • Tom will join the AAI meeting next week at CERN (Jaroslav and Martin will be there as well)
• DUNE/Edinburgh
  • Will be at CERN last week of January
  • Work on Policy package
• Belle II
  • Belle II General meeting at the beginning of February
  • Proposal for Rucio migration to be approved

3 Hot topics

4 Developers roundtable

Rucio_Issue_burn_2020-01-09.png

Rucio_priority_progress_2020-01-09.png

• Issue burn update
• Rucio 1.22 "Green Donkey" priority development followup
  • In Progress
    • Kubernetes [Thomas]
      • Lots of of progress since last week:
      • Integration cluster is running with all the daemons
      • Issues with the Judge-Injector due to memory consumption
      • Still waiting for a cluster with the new network driver (calico)
      • Probes now report to prometheus as well
      • Tested auto-scaling which works as well
      • Lots of log messages in ES - should try to clean this up
    • Reaper 2.0 #2412 [Cedric, Thomas]
      • It's essentially done, a few things need to be improved
      • Query to get list of unlocked replicas needs to be improved
    • OIDC #2612 [Jaroslav]
    • MultiVO #2635 [Ian, Andrew]
      • Further development when Eli starts
      • Need to consider Non-Multi-VO deployments with these changes
    • Repository cleanup #5252 [Martin, Thomas]
      • Almost done
    • RSEMgr v2.0 #3147 [Tomas, Tobi]
      • Upload function moved to upload client
        • Needs to be merged
      • We should do upload/download testing with RSEs in Travis
        • Check with [Mario] to get the docker development also into travis
    • Unification of metadata #3096 [Aris]
      • News next week
    • General update of BB8 #3148 [Tomas, JJ, Dimitrios]
      • No news
      • Some comments on the PR would be helpful
    • Cleanup of Issues [Everyone]
      • Update tickets
  • ToDo
    • Data Pre-Placement [Thomas]
    • Documentation [Martin]
    • rucio.cfg vs config table #2630 [Mario]
    • Versioned history tables need explicit definition #2063 [Martin]
    • Protection of sources too strict in reaper #1637 [Cedric, Thomas]
    • Add file to content history in reaper #37 [Cedric, Thomas]
    • Reaper should stop in case of judge backlog #1578 [Martin]
    • Handling of Archives in Reaper #1431 [Cedric, Thomas, Mario, Martin]
    • Missing features for Archives #1091 [Mario, Martin, Tobi, Tomas]
    • Documentation page listing config table and RSE attributes #2631 [Mario]
    • Operators doc and repo #2636 [Martin]
  • Done
    • Configuration / Permissions / Policy #533 [James]

5 AOB