Rucio Meeting

Martin Barisits (CERN)
Rucio Development Meeting
Zoom Meeting ID
Martin Barisits
Alternative host
Mario Lassnig
Useful links
Join via phone
Zoom URL
    • 1
      • September meeting schedule
        • Summer is over! 
          • Back to weekly meetings
        • Sep 1
        • Sep 8 Jeûne genevois
        • Sep 15
        • Sep 22
        • Sep 29
      • Component lead update
        • Core & Internals
          • Gabriele Fronzé stepped down
        • Authentication & Authorisation
          • Rizart Dona stepped down
        • Deletion
          • Radu Carpa added
        • More updates to follow in the coming weeks
    • 2
      Community News & DevOps roundtable
      • ATLAS
        • Subject DNs in Legacy format
          • Need to figure out how to deal with the RFC-type DNs vs. Legacy DNs
          • IAM-X509 exports only RFC-type DNs
      • CMS
        • 1.29 upgrade
          • Hoped it would fix a problem with large backlog
            • Transfer to tape with archive feature turned on
              • Initial failure was that these timed out
            • Then Rucio submits multi-source with file already exists error
            • Then on next re-submission where rucio should check existing checksums/sizes it does not work either
          • Further try:
            • Set all requests to FAILED for Rucio to resubmit
              • Did not help either
          • Need to clarify if FTS is reporting the correct checksum/data
            • Fermilab FTS is being used, but should be the same as CERN
            • Radu will provide some pointers
        • Issue with DATASETS/CONTAINERS added to CONTAINERS do not follow the ALL grouping
          • Logic was never implemented
          • Will focus on the DATASET attached to CONTAINER case first, more complicated CONTAINER to CONTAINER one later
      • Belle II
        • Upgraded to 1.28.5 for Server and 1.28.7 for daemons
          • Seems to work fine
        • Activated temporary tables as well
          • Should only be the reaper for now; submitter as well if you run it on limited number of RSEs
          • For the rest need to go to 1.29, but need Py3 first
          • Need to check utilisation plots of database
      • DUNE
        • Tape backed RSEs
          • Use surl_algorithm for data placement
        • Preparing for second part of data challenge
          • Planned for September
          • Process data --> Pilots will do rucio uploads
          • More Rucio interactions
      • RAL MutliVO
        • Getting Authentication to work on K8s
    • 3
      Hot topics
    • 4
      Developers roundtable
      • Rucio 1.30 "The Donkeynator" progress tracking
        • In Progress​​
          • Introduce heartbeat handler to all daemons #5443 [Cedric, Dimitrios, Mario]
            • Missing daemons: hermes, atropos, replicarecoverer
              • Hermes done
              • replicarecoverer Christoph will do it, Cedric will double check
              • atropos Dimitrios will check asap
          • Further ingetragtion of CMS-style consistency checking #5438 [Igor]
            • PR for REST interface needs merging
              • --> 1.29.3
            • From Rucio core PoV things are DONE
            • Igor will test daemonless consistency checking then on CMS instance
              • Once done, next steps need to be discussed
                • General purpose utilities probably go to Rucio directly
                • Other things probably to a separate repository for others to use
          • Adherence to rule grouping dependent on order of ATTACH #5251 [Igor]
          • Unable to Delete File DID via Undertaker #5154 [Martin]
            • More complicated than initially thought; still under investigation
        • In Review
        • Done
          • Remove old readthedocs documentation once the new one is fully operational #5446 [Joel, Martin]
        • Todo
          • Refactor the different test workflows to a common one [Ben, Mayank]
          • Create a contributor guide for probes #84 [Dimitrios]
          • Create a server/daemon installation howto #5445 [Joel]
          • Add symlinks for the new daemon names #5131 [Joel]
          • Prepare a clear list of changes which need to be made to use new daemon names #5132 [Joel]
          • Create developers testing guide in the documentation #5452 [Joel, Mayank]
          • Exchange of function-based indices with normal indices #5440 [Mario, Martin, Radu]
          • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
          • Reduce rule tickets to les than 20 [Martin]
          • foreign key error on deleting dids in reaper #5733 [Martin]
          • Rules on containers in state OK but not all the files from the containers have locks #5447 [Martin]
          • Reduce logging of server & daemons #5449 [Mayank]
          • Add python 3.10 tests to CI framework. #5145 [Mayank]
          • Release webUI 2.0 beta version [Mayank]
          • rework prometheus/statsd metric names #5804 [Radu]
          • Re-work/cleanup distance table #5454 [Radu]
          • rework conveyor-throttler #5805 [Radu]
          • Switch to pytest [Radu?]
            • Needs discussion
          • Create documentation page for S3-type storage #5450 [Rizart]
          • Add support for S3 virtual hosted buckets #5451 [Rizart]
            • Test on-premise
            • Ongoing support issue with AWS/Fresno X.509 shenanigans
          • Merge list_dids and list_dids_extended methods #5448 [Rob]
        • Delayed
      • Other topics
    • 5