Rucio Development Meeting

Thursday 6 May 2021, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• May release schedule
  • 1.25.4 today
  • 1.25.5 May 17
  • 1.25.6 May 31
  • Next feature: 1.26 "Donkey league of La Mancha" end of June
• Rucio Community Workshop 2021
  • 28 Sep - 1 Oct 2021
  • To be announced soon
• ESCAPE Rucio Mini Workshop
• Rucio CHEP Paper
• Next two weeks no meeting!

15:05 → 15:15 Community News & DevOps roundtable

• ATLAS
• Fermilab
• Belle II
• Edinburgh / DUNE
• RAL / MultiVO
  • Getting dteam working for SKA
• ESCAPE
  • Finished migration to flux
  • Slight difficulty with gatekeeper
  • Standard ports!
  • Fully configuring openid connect auth
  • Synced subject-ids of users to rucio
  • Authentication does not work for now
    • Querying empty oauth_request table
• SKA
  • Data ingestion workflows
• Regensburg

15:15 → 15:25 Hot topics

15:25 → 15:55 Developers roundtable

Rucio 1.26 "Donkey League of La Mancha" priority followup

• In Progress
  • Test and switch to new rule algorithm #4215 [Martin]
    • Remove old code for 1.26
  • Logging Review #4220 [All Component leads]
  • Re-Factor Transfer workflow [Radu]
    • Re-Structure PR merged
    • Now reviewing the bigger feature PR
  • Discuss and plan mysql5 deprecation [Mario]
    • Alembic stuff finished
    • Go through the code and change some missing things
    • Removal of mysql5 tests needs to be done
    • Documentation requirements table
  • Test SSO Login [Rizart]
    • Studied API and look into connection to CERN SSO
  • Test Import and Export of CRIC JSON [Rizart]
    • ESCAPE needs custom parameters to add to RSEs 
      • Backported feature from ATLAS plugin to mainstream CRIC
      • Some fields in the RSE table, some attributes
      • Discuss with CRIC people to have a more consolidated solution
      • Checked how the different plugins are differentiated
        • Will possibly need a plugin for each community
    • ESCAPE already uses the rucio importer (full json import)
    • Slight modifications needed for importing
  • Auditor overhaul #3437 [Dimitrios, Eric]
  • Hermes2 planning
    • Add column for 1.26 release, but not full functionality
  • Python 3.8 and 3.9 testing #4453 [Ben]
    • Almost done, ready to merge
    • Most of the bugs which appeared are patched
  • Setup.py and requirements overhaul #4456 [Ben]
    • Already started
• Todo
  • Switch to policy-packages as default [James, Martin]
  • Versioned history tables #2063 [Martin]
  • rucio.cfg vs Config table #2630 [Mario] 
  • Cleanup tickets #4463 [*]
• Done
  • Remove webpy endpoints and dependency #4044 [Ben]
  • Migrate documentation to new docusaurus [Martin, Ben, Daniel]
  • Rename reaper2 to reaper #4462 [Martin]
• Delayed
  • QoS #3419 [Mario, Martin, Rizart]

Developer roundtable

• Deletion of last replica, what to do with open datasets
  • If last replica is removed, but dataset is still open
    • No collection replica is removed --> Bug
    • The dataset is not removed (and stays forever) --> Is this wanted?
  • What do other communities think
    • Belle II -> remove it
    • ATLAS -> Different opinion
  • Make it a policy/config switch #XXXX
• FTS Overwrite
  • Ongoing discussion how to fix overwrite issue
• TPC Test and upload client test are good to go
  • Tests are green!

15:55 → 16:00 AOB