Rucio Meeting

Europe/Zurich
Martin Barisits (CERN)
Videoconference
Rucio Development Meeting
Zoom Meeting ID
413496641
Host
Martin Barisits
Alternative hosts
Cedric Serfon, Mario Lassnig, Dimitrios Christidis
Passcode
28849311
Useful links
Join via phone
Zoom URL
    • 15:00 15:05
      News 5m
      • Google Summer of Code 2022
        • Still a few days left!
        • Two Rucio projects
          • Rucio - S3-compatible access interface for analysis facilities
          • Rucio WebUI Revamp
      • Rucio token evolution meeting
        • Please participate in the doodle
        • Mar-03, Mar-10, Mar-17
    • 15:05 15:20
      Community News & DevOps roundtable 15m
      • ATLAS
        • Ongoing query optimization
        • Worker deployments
      • CMS
        • Issue with rules on containers
          • Rule does not affect all datasets
          • Attach order of dataset seems to matter
          • Martin remembers a very similar issue from some years ago
          • Stefan has a similar issue
          • Create ticket with all findings
      • Fermilab/DUNE/Rubin/...
        • Automate transfers between RSEs to generate messages and test
        • TLS change will go into 1.28
        • More complicated due to integration with OpenShift
      • Belle II
        • Moving to next DIRAC release for Py3 migration
          • v7r1 --> v7r2 --> v7r3
      • DIRAC
        • Cosmetic changes done
          • Martin can merge like this (Conveyor test fails, but this is unrelated due to flaky test)
      • ESCAPE
        • S3 endpoint integration
          • Two RSEs in ESCAPE datalake, Rizart trying to get one of them to work
          • Rucio signs URL and passes it to GFAL
          • rucio download works with Google signed URLs, but with S3 it might be a little bit different
          • --> Ticket
      • SKAO
        • Token workflow integration
          • conveyor-poller and finisher not using token workflow but falling back to X509
            • Working now
      • MultiVO/RAL
        • PR for MultiVO issue with S3 endpoints
        • S3 endpoints still not working though
          • --> Same issue as Rizart is having
    • 15:20 15:50
      Python 2.7 deprecation plans 30m
      • Belle II
        • Will move to v7r3 DIRAC version this summer
        • Suggested timeline should not be a problem for Belle II
        • Will move to 1.28 and stay on this even after the summer
          • Extended Support Period might be helpful, but not strictly required
      • FTS
        • Rucio does not use FTS3 clients, does direct REST interface submissions, thus FTS3 client python version does not really matter for Rucio
        • Python3 GFAL bindings
          • Need to test that specifically in Rucio
          • Seems like Python2-gfal package just seems to work fine with Python3 interpreter?
        • Python2 GFAL will still be supported for some time due to community requests
      • ESCAPE
        • Everyone seems to be on Py3 already
      • Rucio Integration Tests for TPC (dev container)
        • Still use FTS3-REST-CLI on python2
          • Python3 version is out, we should change dev container to that one
      • Conclusion
        • 1.29 python 2.7 deprecation is acceptable for all communities, we will go ahead with this plan
        • 1.28 will not be declared an Extended Support release at this point
          • Can still be done later if needed
    • 15:50 15:55
      Developers roundtable 5m
      • Rucio 1.28 "Teenage Mutant Ninja Donkeys" release roadmap followup
        • IN PROGRESS
          • Test framework for VO Specific tests #4797 [Mayank]
            • Flakyness fixed, some more changes to be done, then can start using it for ATLAS, other to follow
          • WebUI 2.0 Architecture #5134 [Mayank]
            • Should be done by March
          • Authentication workflows in WebUI 2.0 #5135 [Mayank]
          • Remove direct DB dependence from the WebUI #4319 [Mayank]
          • Host token evolution meeting [Martin]
            • Waiting for responses and scheduling
          • Clarify, implement and document a way how to include policy packages in containers/kubernetes #5138 [James]
            • Started to write up some guidance about policies
            • Draft PR is there - Please comment!
          • Reduce transfer tickets [Radu]
            • 5 tickets on Radu, 10 more on other people
              • 1 closed, other in progress, 
          • Disentangle fts3 specific code from conveyor poller and receiver #4873 [Radu]
          • Test SSO Login to WebUI [Rizart]
            • Keycloak SSO CERN testing
            • Also started documentation progress for others to repeat the setup
        • IN REVIEW
          • Fix broken Client and Rest API docs view #4992 [Joel]
          • Hermes 2 evolution #4610 [Cedric]
            • First PR is there, missing documentation
            • Dev container is missing Influx, Elastic, etc. would be needed for TESTING
              • Modify dev container
              • Might need some help on container modification
          • BB8 rewrite #4706 [Cedric]
            • Would be good if CMS has a look on this too
              • Eric will check if Bennedikt will look on it
            • Proceed without CMS comment for now, comments can be made later too
          • Create CMS-style consistency checking daemon #5113 [Stefcho]
            • Still responding to review comments
            • Discussion on consistency checking
          • Add MongoDB metadata plugin #5143 [Rob]
        • TODO
          • Reduce rule tickets [Martin]
          • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
          • Add symlinks for the new daemon names #5131 [Joel]
          • Prepare a clear list of changes which need to be made to use new daemon names #5132 [Joel]
          • Clean dead code in the probes (ATLAS and common) #p/83 [Dimitrios, Eric]
            • Work required is clear, but time is the issue
            • Eric provided the list of what is not dead for CMS
          • Create a contributor guide for probes #p/84 [Dimitrios]
        • DONE
          • Create a prototype workflow with temporary tables #5130 [Radu]
          • Support prometheus metrics in daemons and servers #3416 [Radu]
          • Stabilize geoip based replica selection #5133 [Radu]
            • For refresh of the geoip database the creation time is now used
          • Write documentation page about helm-charts and versioning #h/84 [Eric]
          • Host python 2.7 deprecation meeting [Martin]
        • DELAYED
          • Include new operator engine also to json based columns #5144 [Rob]
      • Softcheck for Mypi on CI
        • Will not be final, but gets us started in type-checking our code
      • Python 3.10 tests on CI
        • Dockerfile for Centos 7 problematic
          • Integration suite uses rucio container repository
    • 15:55 16:00
      AOB 5m