Rucio Development Meeting

Thursday 30 Jan 2020, 15:00 → 16:00 Europe/Zurich

4/S-030 (CERN)

Martin Barisits (CERN)

15:00 → 15:10 News

• 3rd Rucio Community Workshop
• GSoC
  • FUSE/Desktop integration
  • Support
  • SWAN Integration
• Release schedule
  • 1.21.6 including the sharding changes
    • Was deployed for ATLAS on Wednesday
    • No issues noticed
  • 1.21.7 coming Monday
  • 1.21.8 on Feb-14 (Irregular date due to my vacation)
  • 1.21.9 on Feb-24
  • 1.22.0rc1 Mar-02
    • Code freeze to be defined
• Paper for IEEE
  • Editors Iteration deadline in a few days
  • Please contact Mario for Overleaf link

15:10 → 15:20 News from the experiments

• CMS
• ATLAS
  • StorM
    • Probably timeout related (Hardcoded 10s) 
    • Function comes back with an error code, but we don't raise an exception
    • More information would be very useful
    • For Monday release increase the timeout
    • Discuss on Tuesday to switch away from davix-http
• Belle-2
• DUNE/Edinburgh
  • Customisation for VO-specific packages
    • construct_surl methods need more development
  • Metadata catalog of DUNE
• ESCAPE
  • GSoC proposal for SWAN ready
  • Issue with rucio upload client
    • Sometimes error when stat file
    • Only seen recently in the last month
• LDMX (David)
  • Dark matter experiment in the US asking for a DDM solution
  • Setting up docker demo worked, setting up full docker environment is lacking documentation
    • Difficult part is the auth server
    • Need more documentation

15:20 → 15:30 Hot topics

15:30 → 15:55 Developers roundtable

Rucio_Issue_burn_2020-01-30.png

Rucio_priority_progress_2020-01-30.png

• Issue burn update
• Rucio 1.22 "Green Donkey" priority development followup
  • In Progress
    • Kubernetes [Thomas]
      • No news on the network controller
    • OIDC #2612 [Jaroslav]
      • PR in a very good state
    • MultiVO #2635 [Ian, Andrew]
      • Further development when Eli starts
      • Need to consider Non-Multi-VO deployments with these changes
    • RSEMgr v2.0 #3147 [Tomas, Tobi]
      • Waiting on review & merge of one iteration
    • Unification of metadata #3096 [Aris]
      • Rucio core part is mostly done
      • API/CLI work needs to be done
        • Re-use existing calls
  • ToDo
    • Documentation [Martin]
    • rucio.cfg vs config table #2630 [Mario]
    • Versioned history tables need explicit definition #2063 [Martin]
    • Add file to content history in reaper #37 [Cedric, Thomas]
      • Getting increasingly problematic since we cannot track if there are problems
    • Reaper should stop in case of judge backlog #1578 [Martin]
    • Handling of Archives in Reaper #1431 [Cedric, Thomas, Mario, Martin]
    • Missing features for Archives #1091 [Mario, Martin]
    • Documentation page listing config table and RSE attributes #2631 [Mario]
    • Operators doc and repo #2636 [Martin]
      • Dimitrios hopes to start an effort mid-february
        • Some ATLAS parts might be required to stay confidential
  • Done
    • Repository cleanup #5252 [Martin, Thomas]
    • Configuration / Permissions / Policy #533 [James]
    • Cleanup of Issues [Everyone]
    • Reaper 2.0 #2412 [Cedric, Thomas]
    • Protection of sources too strict in reaper #1637 [Cedric, Thomas]
• Auditor
  • Contacted by CMS who want to continue on Auditor work
  • Dimitrios and Thomas will try to give pointers, probably should re-write some parts of the auditor into an auditor2
• Third party copy read and write needs to be enabled [Martin, Cedric]
  • Duplicate the information for both operations in the probe for now
  • Need to iterate with AGIS to get TPC_write and _read in there
• CRIC progress
• Review probes
  • Dimitrios needs some help with reviewing the probes, since he is overloaded
  • Thomas will help with reviewing
• RPG development
  • Tomas will write document about project definition for RucioRPG

15:55 → 16:00 AOB