Rucio Development Meeting

Thursday 16 May 2019, 15:00 → 16:30 Europe/Zurich

4/S-056 (CERN)

Martin Barisits (CERN)

15:00 → 15:10 News

• Rucio Coding Camp 2019
  • Please register
• Code Freeze for 1.20.0 on May 23rd 23:59 CEST

15:10 → 15:30 News from the experiments

• ATLAS
  • Nothing to report
• CMS
  • Timeout in Apache module
    • Way to monitor better?
    • Not obvious, increase log level and try to follow
    • Try nginx instead of apache?
  • Kubernetes issues
    • Also a bit on doma instance
  • Enabled the Oracle Jobs (Collection replicas)
    • Works well
  • Looking into further procedures
• ESCAPE
• STFC
• SKA

15:30 → 15:50 Hot topics

15:50 → 16:20 Developers roundtable

• 1.20.0 "Wonder Donkey" priority followup
  • Rucio mover
    • pcacheand storm symlinks
    • Some problems with pilot2
    • pcache has to be changed quite a bit, will be added later
    • storm protocol works when specified directly
      • Server filters storm protocol if not specified directly
      • Client needs to load protocols fail-save
  • Python 2.6 drop
    • Martin will do it soon #2460
    • Almost done
  • Reaper
    • Partitioning/Full heartbeat support of reaper #529
      • Need to protect storage 
        • Probably fine for dCache without protection
        • Probably not fine for all the others
        • One idea is to extend heartbeat to report the RSE the thread is working on
    • Relax protection of sources in reaper #1637
    • Fill contents history when deleting last replica #37
    • Archive Handling in the reaper #1431
    • Proposal: Google Doc
    • PR for Heartbeat is there
    • Cedric & Thomas looking into it next week
  • Get rid of unused dockers
    • Base client container
      • Notify clients next week and then remove community specific containers
      • Need to check if it is automatically build
      • Build an ATLAS rucio client based on the BASE Rucio client
      • Once done, tell XDC and Eiscat3D how to do theirs based on the BASE client
  • Migrate probes to separate repo #1638
    • PR including history is done
    • Will be switched on ATLAS machines with 1.20
  • Better way to deal with configuration/permissions #533
    • Mario wants to help!
    • Moved to 1.21 release
  • Transparent handling of archives with rules #1091
    • Re-use of list_replicas code not really possible, needs to adressed se
  • Daemon to re-order tape requests (throttler) #2220
    • Added volume check
    • Problems with MySQL due to Window function
    • Will force to require mysql 8 for window functions
    • Will need a separate meeting to discuss the reading (writing is covered)
    • For reading mode --> Need to add source_rse_id then Hannes development works already
    • PR is there, needs to be reviewed
    • Martin will look into filling of src_rse_id when queuing requests
• Question Natalia
  • Re-Submission of failed source
    • Is FTS Multi Source job used?
      • Not really, as only available in single job submission, not bulk;
    • What happens when site goes into downtime?
      • Rucio cancels jobs and resubmits
• Pilot
  • Issue with propagation of error messages
  • Rucio sending the traces difficult, due to loss of information from pilot
  • When rucio download fails --> give error information to pilot --> Pilot sends traces based on this
  • Invite ATLAS Pilot people to ATLAS DevOps people next week
• WebUI now available for helm-charts!
• SSO development for GSOC student
  • Ramping up
•  

16:20 → 16:30 AOB

• Summer student at FNAL for CRIC->Rucio import
  • Development should go to Rucio core repo when possible