Rucio Meeting

Martin Barisits (CERN)
Rucio Development Meeting
Zoom Meeting ID
Martin Barisits
Alternative host
Mario Lassnig
Useful links
Join via phone
Zoom URL
    • 3:00 PM 3:05 PM
      News 5m
      • Next week python 2.7 deprecation discussion
      • Google Summer of Code reminder
        • WebUI - Mayank
        • S3FrontEnd - Mario
    • 3:05 PM 3:20 PM
      Community News & DevOps roundtable 15m
      • ATLAS
        • Loadbalancer updates
          • MaxConn limitation
        • Poller and Reaper optimization to reduce DB load
      • CMS
      • Fermilab/DUNE/Rubin/etc
      • Belle II
      • DUNE/Edinburgh
      • DIRAC
        • MultiVO changes done
          • Cedric reviewed and approved
          • Martin will merge as a patch (most likely)
      • SKAO
        • Including Token enabled storages
        • Workflow with FTS and tokens enabled
        • Found some bugs in OIDC workflow in Rucio, needs some fixes
        • Documentation
          • Some missing information
      • MultiVO/STFC
        • Adding an S3 endpoint
          • S3 on the client not working?
            • get_signed_url pieces of the code working
            • Upload does not work
            • Needed to hardcode rse_id
          • James can have a look
          • Mario had a UnitTest to test, and it seemed to work
          • API/Client code asks for RSEName, Protocol, action, VO defaults to "def"
    • 3:20 PM 3:25 PM
      Hot topics 5m
    • 3:25 PM 3:55 PM
      Developers roundtable 30m
      • Rucio 1.28 "Teenage Mutant Ninja Donkeys" release roadmap followup
        • IN PROGRESS
          • Fix broken Client and Rest API docs view #4992 [Joel]
          • Test framework for VO Specific tests #4797 [Mayank]
            • Initial private PR is there running the VO tests for ATLAS+BelleII
            • Some flaky tests (unrelated) still need fixes
            • Then PR is ready to go
            • Go ahead with ATLAS tests, merge it, and Belle II tests can be activated later
          • Authentication workflows in WebUI 2.0 #5135 [Mayank]
          • WebUI 2.0 Architecture #5134 [Mayank]
            • Remove direct DB dependence from the WebUI #4319 [Mayank]
            • Should be done by March
          • Host python 2.7 deprecation meeting [Martin]
          • Host token evolution meeting [Martin]
            • Will send out eMails by tomorrow
          • Clarify, implement and document a way how to include policy packages in containers/kubernetes #5138 [James]
            • Started to write up some guidance about policies
            • Draft PR is there - Please comment!
          • Reduce transfer tickets [Radu]
            • 2 to be closed, 1 more to be opened
            • 5 tickets on Radu, 10 more on other people
              • Nobody answered!?
          • Disentangle fts3 specific code from conveyor poller and receiver #4873 [Radu]
        • IN REVIEW
          • Hermes 2 evolution #4610 [Cedric]
            • First PR is there, missing documentation
            • Dev container is missing Influx, Elastic, etc. would be needed for TESTING
              • Modify dev container
              • Might need some help on container modification
          • BB8 rewrite #4706 [Cedric]
            • Would be good if CMS has a look on this too
              • Eric will check if Bennedikt will look on it
          • Create CMS-style consistency checking daemon #5113 [Stefcho]
            • Still responding to review comments
            • Discussion on consistency checking
        • TODO
          • Reduce rule tickets [Martin]
          • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
          • Add symlinks for the new daemon names #5131 [Joel]
          • Prepare a clear list of changes which need to be made to use new daemon names #5132 [Joel]
          • Clean dead code in the probes (ATLAS and common) #p/83 [Dimitrios, Eric]
            • Work required is clear, but time is the issue
            • Eric provided the list of what is not dead for CMS
          • Create a contributor guide for probes #p/84 [Dimitrios]
          • Add MongoDB metadata plugin #5143 [Rob]
          • Test SSO Login to WebUI [Rizart]
        • DONE
          • Create a prototype workflow with temporary tables #5130 [Radu]
          • Support prometheus metrics in daemons and servers #3416 [Radu]
          • Stabilize geoip based replica selection #5133 [Radu]
            • For refresh of the geoip database the creation time is now used
          • Write documentation page about helm-charts and versioning #h/84 [Eric]
        • DELAYED
          • Include new operator engine also to json based columns #5144 [Rob]
      • GFALv2 module is still in protocols
        • Remove for 1.28
      • Python 3 type annotations
        • Needs some explenation how to do it
          • Part of development guide
        • Would need a higher percentage of current code to actual use it
          • People mostly copy&paste these
          • Have reviewers to remind people about it 
          • Possibly do this in multiple stages
            • Update larger parts of the code first
            • Start reminding people, but not require it yet
            • Only require later
        • mypi checking would still need to be added
    • 3:55 PM 4:00 PM
      AOB 5m