Rucio Meeting

Europe/Zurich
Martin Barisits (CERN)
Videoconference
Rucio Development Meeting
Zoom Meeting ID
413496641
Host
Martin Barisits
Alternative hosts
Mario Lassnig, Cedric Serfon, Dimitrios Christidis
Passcode
28849311
Useful links
Join via phone
Zoom URL
    • 3:00 PM 3:05 PM
      News 5m
    • 3:05 PM 3:20 PM
      Community News & DevOps roundtable 15m
      • ATLAS
        • Commercial cloud "exceptions" in core list replicas so that they work with xcaches
          • Needs to be made configurable
        • Update interaction between preparer and submitter in 1.29.5
          • Works well in production for ATLAS
          • Difference
            • If preparer is used it will calculate the selection of the source in preparer, instead of submitter
            • More intelligent selection in preparer, which is important for transfertool (FTS, Globus) selection
      • CMS
        • ActiveMQ: Very high number of connections to brokers
          • Yuyi or Eric will have a closer look 
          • Issue is not visible in ATLAS - might be related to not using SSL
          • Possible on client side: 
            • Connection being closed but not re-used?
          • Connection limits on kubernetes node side?
      • Fermilab/DUNE/...
        • Testing policy packages for DUNE
        • Trying to setup prometheus monitoring (Difficult on open shift)
        • RUBIN: Working on kubernetes deployment
      • DUNE
        • Request: Adding non-RSE distances to Rucio
      • ESCAPE/EOSC-Future
        • Adding S3 RSE to cluster
      • RAL/MultiVO
        • Trying to get Auth server to work in k8s
    • 3:20 PM 3:25 PM
      Hot topics 5m
    • 3:25 PM 3:55 PM
      Developers roundtable 30m
      • Rucio 1.30 "The Donkeynator" progress tracking
        • In Progress​​
          • Introduce heartbeat handler to all daemons #5443 [Cedric]
            • Missing daemons:  replicarecoverer
          • Create a contributor guide for probes #84 [Dimitrios]
          • Adherence to rule grouping dependent on order of ATTACH #5251 [Igor]
          • Exchange `Tombstone` function-based indices with normal indices #5440 [Mario, Martin, Radu]
          • Create a server/daemon installation howto #5445 [Joel, Mayank]
            • To check with Joel
          • Add python 3.10 tests to CI framework. #5145 [Mayank]
            • Splitting DockerFiles into multiple stages with python venv with supported versions of python
            • Should be there next week
          • Unable to Delete File DID via Undertaker #5154 [Martin]
            • More complicated than initially thought; still under investigation
          • Rules on containers in state OK but not all the files from the containers have locks #5447 [Martin]
          • Release webUI 2.0 beta version [Mayank]
            • Auth mostly complete
              • New WebUI is pure Browser application
              • Oauth2 with client auth workflow cannot be used for pure browser applications, as it would expose the client secret
              • PKCE auth workflow will solve this
                • IdP needs to support this
                • Need to confirm if this is possible IAM, and others
            • GSOC student will work on high priority pages in the last 4 weeks
          • rework conveyor-throttler #5805 [Radu]
          • Switch to pytest [Radu?]
        • In Review
          • UI Methods to declare replicas missing #5392
        • Done
          • Further ingetragtion of CMS-style consistency checking #5438 [Igor]
          • Remove old readthedocs documentation once the new one is fully operational #5446 [Joel, Martin]
        • Todo
          • Create developers testing guide in the documentation #5452 [Joel, Mayank]
          • foreign key error on deleting dids in reaper #5733 [Martin]
          • Re-work/cleanup distance table #5454 [Radu]
          • Merge list_dids and list_dids_extended methods #5448 [Rob]
        • Delayed
          • Refactor the different test workflows to a common one [Ben, Mayank]
          • Add symlinks for the new daemon names #5131 [Joel]
          • Prepare a clear list of changes which need to be made to use new daemon names #5132 [Joel]
          • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
          • Reduce rule tickets to les than 20 [Martin]
          • Reduce logging of server & daemons #5449 [Mayank
          • rework prometheus/statsd metric names #5804 [Radu]
          • Create documentation page for S3-type storage #5450 [Rizart]
          • Add support for S3 virtual hosted buckets #5451 [Rizart]
            • Test on-premise
            • Ongoing support issue with AWS/Fresno X.509 shenanigans
      • Other topics
    • 3:55 PM 4:00 PM
      AOB 5m