Rucio Development Meeting

Thursday 28 Mar 2019, 15:00 → 16:30 Europe/Zurich

4/S-056 (CERN)

Martin Barisits (CERN)

15:00 → 15:10 News

• Google summer of code submission starting
  • Two Rucio projects
• One CERN summer student
• Waiting for ATTRACT responses
  • One rejected; Other one waiting

15:10 → 15:30 News from the experiments

• ATLAS
  • Had to fix a few issues with the recent True/False/0/1 changes
  • One puppet manifest flip/flopping
    • Possibly caused by Tracer manifest
• CMS
  • checksums crc: How to handle?
    • If no checksum is given, rucio can ask fts to skip checksum check
    • Possibility to move to generic checksum column with Rucio 1.20
      • Ticket added #2410
• DUNE
  • Objectstore development
• STFC
  • Elastic search setup progressing

15:30 → 15:50 Hot topics

• Release policy
  • Proposal
  • Please put comments until Monday, then it will be sent out to community

15:50 → 16:20 Developers roundtable

• 1.20.0 "Wonder Donkey" priority followup
  • Rucio mover
    • pcache and storm symlinks
    • Some problems with pilot2
    • pcache has to be changed quite a bit, will be added later
  • Python 2.6 drop
    • Will be done shortly before release
  • Reaper
    • Partitioning/Full heartbeat support of reaper #529
      • Need to protect storage 
        • Probably fine for dCache without protection
        • Probably not fine for all the others
        • One idea is to extend heartbeat to report the RSE the thread is working on
    • Relax protection of sources in reaper #1637
    • Fill contents history when deleting last replica #37
    • Archive Handling in the reaper #1431
    • Will send out proposal tomorrow
  • Get rid of unused dockers
    • Base client container
      • Notify clients next week and then remove community specific containers
      • Need to check if it is automatically build
    • Oracle Instaclient is in the container - License allows this!
  • Migrate probes to separate repo #1638
    • PR including history is done
    • Will be switched on ATLAS machines with 1.20
  • Better way to deal with configuration/permissions #533
  • Transparent handling of archives with rules #1091
    • Re-use of list_replicas code not really possible, needs to adressed se
  • Daemon to re-order tape requests (throttler) #2220
    • Throttler can release even if it goes over limited
    • Deadline for Reading, to release anyway.
    • Volume check --> needs to be adapted in the rse_transfer_limits
    • Check with FTS team about workflow
• High priority: List replicas needs to resolve parent did recursively #2217
  • Submitted by Mario; Uncovered a bug
  • Need to check how we are exposed for loops in container definitions!
    • Needs to be called only in adding CONTAINER to CONTAINER
• FTS provides functionality to block duplicate submission
  • if error is raised, we submit 1 by 1
•  

16:20 → 16:30 AOB

• CHEP Paper
  • ATLAS deadline
  • Mario will draft something and send around