Rucio Development Meeting

Thursday 5 Mar 2020, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

1 News

• Rucio meetings Vidyo-Only
  • Until further notice
• Release schedule
  • 1.22 "Green Donkey"
    • Code Freeze on Mar-16
    • RC1 on Mar-18
    • Final ~Mar-24
  • 1.21.11 patch release
• Rucio Workshop
  • COVID-19 Travel advisory
• Rucio Coding Camp
• Meeting Schedule
  • Mar-12   Rucio Workshop
  • Mar-19   Regular Meeting
  • Mar-26   Might be CANCELLED

2 News from the experiments

• ATLAS
  • Deletion issue with contents_history
    • Being investigated
    • Understood and being fixed
  • FTS Poller issue
    • Under investigation
    • Found partitioning issue, should be unrelated
• CMS
• Belle II
  • Testing DIRAC-Rucio client
  • Scaling tests
• RAL/Multi-VO
  • New member of staff for 6-month placement
    • Will be working on Multi-VO
  • Monitoring transfer activity
• DUNE/Edinburgh
• Janusz/DIRAC
  • Rucio instance running
  • Some workflows already working
• LIGO/VIRGO/KAGRA
  • First successful transfer from KAGRA (Tokyo) to LIGO (Caltech)!

3 Hot topics

4 Developers roundtable

Rucio_Issue_burn_2020-03-05.png

Rucio_priority_progress_2020-03-05.png

• Issue burn update
• Rucio 1.22 "Green Donkey" priority development followup
  • In Progress
    • Kubernetes [Thomas]
      • Waiting on the new network controller
      • It's available, but from testing it was not faster than before
      • CERN IT: Problem with multi-zone deployments, network driver goes to fallback mode
    • MultiVO #2635 [Ian, Eli, Patrick]
      • Open PR works fine now; Will be merged to coming feature release
      • Andrew prepared a few other things which can be improved and submitted incrementally (probably)
    • RSEMgr v2.0 #3147 [Tomas, Tobi]
      • Upload migration PR is open since some time, probably needs to be re-submitted for testing
    • Unification of metadata #3096 [Aris]
      • Rucio core part is mostly done
      • API/CLI work needs to be done
        • Re-use existing calls
    • Add file to content history in reaper #37 [Cedric, Thomas]
      • Done, but one bug to fix due to contents_history partitioning
  • ToDo
    • Documentation [Martin]
    • rucio.cfg vs config table #2630 [Mario]
    • Versioned history tables need explicit definition #2063 [Martin]
    • Handling of Archives in Reaper #1431 [Cedric, Thomas, Mario, Martin]
    • Missing features for Archives #1091 [Mario, Martin]
    • Documentation page listing config table and RSE attributes #2631 [Mario]
    • Operators doc and repo #2636 [Martin]
      • Dimitrios hopes to start an effort mid-february
        • Some ATLAS parts might be required to stay confidential
  • Done
    • Repository cleanup #5252 [Martin, Thomas]
    • Configuration / Permissions / Policy #533 [James]
    • Cleanup of Issues [Everyone]
    • Reaper 2.0 #2412 [Cedric, Thomas]
    • Protection of sources too strict in reaper #1637 [Cedric, Thomas]
    • OIDC #2612 [Jaroslav]
    • Reaper should stop in case of judge backlog #1578 [Martin]
• Auditor
  • Contacted by CMS who want to continue on Auditor work
  • Dimitrios and Thomas will try to give pointers, probably should re-write some parts of the auditor into an auditor2
• Third party copy read and write needs to be enabled [Martin, Cedric, Dimitrios]
  • Duplicate the information for both operations in the probe for now
  • Need to iterate with AGIS to get TPC_write and _read in there
  • No news
• CRIC progress [Panos]
• Review probes [Thomas, Dimitrios]
  • Dimitrios needs some help with reviewing the probes, since he is overloaded
  • Thomas will help with reviewing
  • No news :-)
• RPG development [Tomas]
  • Tomas will write document about project definition for RucioRPG
  • Needs more discussion
• Issue with Reaper2 which unnecessarily deletes secondary replicas due to replicas in wrong state
• staging_startged_at staging_finished_at working now?
  • Need to check

5 AOB

• Workshop next week!