Rucio Development Meeting

Thursday 23 May 2019, 15:00 → 16:30 Europe/Zurich

4/S-056 (CERN)

Martin Barisits (CERN)

15:00 → 15:10 News

• DIRAC Rucio Bridge
  • Discussed and planned at the 9th DIRAC user workshop
  • Will be implemented via a RucioCatalog plugin in DIRAC
  • Work done by Imperial College
• 1.20.0 "Wonder Donkey" Code Freeze at 23:59 CEST
  • 1.20.0.rc1 build on Monday
• Next week no Rucio Development Meeting (Holiday in Geneva)
• Next Development meeting on June 6th
  • Rucio 1.21.0 "Donkeys of the Galaxy" Roadmap planning

15:10 → 15:30 News from the experiments

• CMS
  • Session issue is understood; Working on the test database
  • Mysterious timeouts issue not understood yet
    • Buildup of the wsgi container takes a while (10-15s)
    • Request takes exactly 60s
      • Looks like a 60s timeout somewhere
      • Possibility: IPv6 retry
• ATLAS
  • Nothing
• ESCAPE
• LIGO
• Multi-VO
  • Transparent changes still

15:30 → 15:50 Hot topics

15:50 → 16:20 Developers roundtable

• 1.20.0 "Wonder Donkey" priority followup
  • Rucio mover
    • pcacheand storm symlinks
    • Some problems with pilot2
    • pcache has to be changed quite a bit, will be added later
    • storm protocol works when specified directly
      • Server filters storm protocol if not specified directly
      • Client needs to load protocols fail-save
  • Python 2.6 drop
    • Martin will do it soon #2460
    • Almost done
  • Reaper
    • Partitioning/Full heartbeat support of reaper #529
      • Need to protect storage 
        • Probably fine for dCache without protection
        • Probably not fine for all the others
        • One idea is to extend heartbeat to report the RSE the thread is working on
    • Relax protection of sources in reaper #1637
    • Fill contents history when deleting last replica #37
    • Archive Handling in the reaper #1431
    • Proposal: Google Doc
    • PR for Heartbeat is there
    • Cedric & Thomas looking into it next week
  • Get rid of unused dockers
    • Base client container
      • Notify clients next week and then remove community specific containers
      • Need to check if it is automatically build
      • Build an ATLAS rucio client based on the BASE Rucio client
      • Once done, tell XDC and Eiscat3D how to do theirs based on the BASE client
  • Migrate probes to separate repo #1638
    • PR including history is done
    • Will be switched on ATLAS machines with 1.20
  • Better way to deal with configuration/permissions #533
    • Mario wants to help!
    • Moved to 1.21 release
  • Transparent handling of archives with rules #1091
    • Re-use of list_replicas code not really possible, needs to adressed se
  • Daemon to re-order tape requests (throttler) #2220
    • Added volume check
    • Problems with MySQL due to Window function
    • Will force to require mysql 8 for window functions
    • Will need a separate meeting to discuss the reading (writing is covered)
    • For reading mode --> Need to add source_rse_id then Hannes development works already
    • PR is there, needs to be reviewed
    • Martin will look into filling of src_rse_id when queuing requests
• Transfers with CTA
  • Workflow in Rucio
  • Flag for source RSE
    • RSE_USED_FOR_STAGING (or similar)
  • Which process creates the intermediate replica
    • Submitter
  • Need to understand if there are two callbacks or only one
    • For finisher to figure out
  • Colocation metadata
    • Should be told to CTA per metadata flag
• CRIC json import
  • json space reporting field?
  • json looks rather good now
  • Distances missing, needs to be discussed
  • CRIC Discussion together with CMS
• Authentication discussion

16:20 → 16:30 AOB

• Next week no meeting!
• 1.21.0 roadmap discussion in 2 weeks
• Twitter account
  • Rucio twitter account
    • Mario creates it