Rucio Meeting

Martin Barisits (CERN)
Rucio Development Meeting
Zoom Meeting ID
Martin Barisits
Alternative hosts
Mario Lassnig, Cedric Serfon, Dimitrios Christidis
Useful links
Join via phone
Zoom URL
    • 3:00 PM 3:05 PM
      News 5m
    • 3:05 PM 3:20 PM
      Community News & DevOps roundtable 15m
      • ATLAS
        • Commercial cloud "exceptions" in core list replicas so that they work with xcaches
          • Needs to be made configurable
        • Update interaction between preparer and submitter in 1.29.5
          • Works well in production for ATLAS
          • Difference
            • If preparer is used it will calculate the selection of the source in preparer, instead of submitter
            • More intelligent selection in preparer, which is important for transfertool (FTS, Globus) selection
      • CMS
        • ActiveMQ: Very high number of connections to brokers
          • Yuyi or Eric will have a closer look 
          • Issue is not visible in ATLAS - might be related to not using SSL
          • Possible on client side: 
            • Connection being closed but not re-used?
          • Connection limits on kubernetes node side?
      • Fermilab/DUNE/...
        • Testing policy packages for DUNE
        • Trying to setup prometheus monitoring (Difficult on open shift)
        • RUBIN: Working on kubernetes deployment
      • DUNE
        • Request: Adding non-RSE distances to Rucio
      • ESCAPE/EOSC-Future
        • Adding S3 RSE to cluster
      • RAL/MultiVO
        • Trying to get Auth server to work in k8s
    • 3:20 PM 3:25 PM
      Hot topics 5m
    • 3:25 PM 3:55 PM
      Developers roundtable 30m
      • Rucio 1.30 "The Donkeynator" progress tracking
        • In Progress​​
          • Introduce heartbeat handler to all daemons #5443 [Cedric]
            • Missing daemons:  replicarecoverer
          • Create a contributor guide for probes #84 [Dimitrios]
          • Adherence to rule grouping dependent on order of ATTACH #5251 [Igor]
          • Exchange `Tombstone` function-based indices with normal indices #5440 [Mario, Martin, Radu]
          • Create a server/daemon installation howto #5445 [Joel, Mayank]
            • To check with Joel
          • Add python 3.10 tests to CI framework. #5145 [Mayank]
            • Splitting DockerFiles into multiple stages with python venv with supported versions of python
            • Should be there next week
          • Unable to Delete File DID via Undertaker #5154 [Martin]
            • More complicated than initially thought; still under investigation
          • Rules on containers in state OK but not all the files from the containers have locks #5447 [Martin]
          • Release webUI 2.0 beta version [Mayank]
            • Auth mostly complete
              • New WebUI is pure Browser application
              • Oauth2 with client auth workflow cannot be used for pure browser applications, as it would expose the client secret
              • PKCE auth workflow will solve this
                • IdP needs to support this
                • Need to confirm if this is possible IAM, and others
            • GSOC student will work on high priority pages in the last 4 weeks
          • rework conveyor-throttler #5805 [Radu]
          • Switch to pytest [Radu?]
        • In Review
          • UI Methods to declare replicas missing #5392
        • Done
          • Further ingetragtion of CMS-style consistency checking #5438 [Igor]
          • Remove old readthedocs documentation once the new one is fully operational #5446 [Joel, Martin]
        • Todo
          • Create developers testing guide in the documentation #5452 [Joel, Mayank]
          • foreign key error on deleting dids in reaper #5733 [Martin]
          • Re-work/cleanup distance table #5454 [Radu]
          • Merge list_dids and list_dids_extended methods #5448 [Rob]
        • Delayed
          • Refactor the different test workflows to a common one [Ben, Mayank]
          • Add symlinks for the new daemon names #5131 [Joel]
          • Prepare a clear list of changes which need to be made to use new daemon names #5132 [Joel]
          • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
          • Reduce rule tickets to les than 20 [Martin]
          • Reduce logging of server & daemons #5449 [Mayank
          • rework prometheus/statsd metric names #5804 [Radu]
          • Create documentation page for S3-type storage #5450 [Rizart]
          • Add support for S3 virtual hosted buckets #5451 [Rizart]
            • Test on-premise
            • Ongoing support issue with AWS/Fresno X.509 shenanigans
      • Other topics
    • 3:55 PM 4:00 PM
      AOB 5m