Rucio Development Meeting

Thursday 4 Apr 2019, 15:00 → 16:30 Europe/Zurich

4/S-056 (CERN)

Martin Barisits (CERN)

15:00 → 15:10 News

• GSOC proposal deadline next week
  • Several good submissions
• Andrew Lister at CERN for 2 weeks to kick-start Multi-VO work
• Rucio Coding Camp 2019
  • Participate in the doodle
• Global quota development
  • To be added to 1.21 release line

15:10 → 15:30 News from the experiments

• ATLAS
  • Lots of CTA tape discussions going on at the moment
    • Planning some stress tests for CTA
    • Evaluating tape families for ATLAS
• DUNE
  • Ramping up and getting people included
• ESCAPE
  • Setting up test-data-lake, connecting users to the lake

15:30 → 15:50 Hot topics

• Discussion of Release Policy proposal
  • Keep the versioning like it is but have a page better describing our compatibility policy
  • Have a test testing new/old clients against different server versions?
• Discussion of Reaper 2.0 design proposal
  • Will go for Proposal 1
  • Work can be split in 3 parts:
    • Cedric, Thomas, Martin?

15:50 → 16:20 Developers roundtable

• 1.20.0 "Wonder Donkey" priority followup
  • Rucio mover
    • pcache and storm symlinks
    • Some problems with pilot2
    • pcache has to be changed quite a bit, will be added later
  • Python 2.6 drop
    • Martin will do it soon
  • Reaper
    • Partitioning/Full heartbeat support of reaper #529
      • Need to protect storage 
        • Probably fine for dCache without protection
        • Probably not fine for all the others
        • One idea is to extend heartbeat to report the RSE the thread is working on
    • Relax protection of sources in reaper #1637
    • Fill contents history when deleting last replica #37
    • Archive Handling in the reaper #1431
    • Will send out proposal tomorrow
  • Get rid of unused dockers
    • Base client container
      • Notify clients next week and then remove community specific containers
      • Need to check if it is automatically build
      • Build an ATLAS rucio client based on the BASE Rucio client
      • Once done, tell XDC and Eiscat3D how to do theirs based on the BASE client
    • Oracle Instaclient is in the container - License allows this!
  • Migrate probes to separate repo #1638
    • PR including history is done
    • Will be switched on ATLAS machines with 1.20
  • Better way to deal with configuration/permissions #533
    • Mario wants to help!
  • Transparent handling of archives with rules #1091
    • Re-use of list_replicas code not really possible, needs to adressed se
  • Daemon to re-order tape requests (throttler) #2220
    • Added volume check
    • Problems with MySQL due to Window function
    • Will force to require mysql 8 for window functions
• Alembic upgrade scripts
  • Change engine to do CHECK_CONSTRAINTS instead of ENUM types
  • Needs a change to most alembic upgrade scripts
  • Put in the readme about ENUM in older versions
  • --> #2436

16:20 → 16:30 AOB

• Security considerations:
  • Authentication for Rucio to Storage, FTS, Users (Full stack)
  • Mario will present a proposal (Also to get rid of VOMS) in one of the next meetings
  • VOMS decommissioning before Run4