Rucio Meeting

Europe/Zurich
Martin Barisits (CERN)
Videoconference
Rucio Development Meeting
Zoom Meeting ID
413496641
Host
Martin Barisits
Alternative hosts
Cedric Serfon, Mario Lassnig, Dimitrios Christidis
Passcode
28849311
Useful links
Join via phone
Zoom URL
    • 1
      News
      • September meeting schedule
        • Summer is over! 
          • Back to weekly meetings
        • Sep 1
        • Sep 8 Jeûne genevois
        • Sep 15
        • Sep 22
        • Sep 29
      • Component lead update
        • Core & Internals
          • Gabriele Fronzé stepped down
        • Authentication & Authorisation
          • Rizart Dona stepped down
        • Deletion
          • Radu Carpa added
        • More updates to follow in the coming weeks
    • 2
      Community News & DevOps roundtable
      • ATLAS
        • Subject DNs in Legacy format
          • Need to figure out how to deal with the RFC-type DNs vs. Legacy DNs
          • IAM-X509 exports only RFC-type DNs
      • CMS
        • 1.29 upgrade
          • Hoped it would fix a problem with large backlog
            • Transfer to tape with archive feature turned on
              • Initial failure was that these timed out
            • Then Rucio submits multi-source with file already exists error
            • Then on next re-submission where rucio should check existing checksums/sizes it does not work either
          • Further try:
            • Set all requests to FAILED for Rucio to resubmit
              • Did not help either
          • Need to clarify if FTS is reporting the correct checksum/data
            • Fermilab FTS is being used, but should be the same as CERN
            • Radu will provide some pointers
        • Issue with DATASETS/CONTAINERS added to CONTAINERS do not follow the ALL grouping
          • Logic was never implemented
          • Will focus on the DATASET attached to CONTAINER case first, more complicated CONTAINER to CONTAINER one later
      • Belle II
        • Upgraded to 1.28.5 for Server and 1.28.7 for daemons
          • Seems to work fine
        • Activated temporary tables as well
          • Should only be the reaper for now; submitter as well if you run it on limited number of RSEs
          • For the rest need to go to 1.29, but need Py3 first
          • Need to check utilisation plots of database
      • DUNE
        • Tape backed RSEs
          • Use surl_algorithm for data placement
        • Preparing for second part of data challenge
          • Planned for September
          • Process data --> Pilots will do rucio uploads
          • More Rucio interactions
      • RAL MutliVO
        • Getting Authentication to work on K8s
    • 3
      Hot topics
    • 4
      Developers roundtable
      • Rucio 1.30 "The Donkeynator" progress tracking
        • In Progress​​
          • Introduce heartbeat handler to all daemons #5443 [Cedric, Dimitrios, Mario]
            • Missing daemons: hermes, atropos, replicarecoverer
              • Hermes done
              • replicarecoverer Christoph will do it, Cedric will double check
              • atropos Dimitrios will check asap
          • Further ingetragtion of CMS-style consistency checking #5438 [Igor]
            • PR for REST interface needs merging
              • --> 1.29.3
            • From Rucio core PoV things are DONE
            • Igor will test daemonless consistency checking then on CMS instance
              • Once done, next steps need to be discussed
                • General purpose utilities probably go to Rucio directly
                • Other things probably to a separate repository for others to use
          • Adherence to rule grouping dependent on order of ATTACH #5251 [Igor]
          • Unable to Delete File DID via Undertaker #5154 [Martin]
            • More complicated than initially thought; still under investigation
        • In Review
        • Done
          • Remove old readthedocs documentation once the new one is fully operational #5446 [Joel, Martin]
        • Todo
          • Refactor the different test workflows to a common one [Ben, Mayank]
          • Create a contributor guide for probes #84 [Dimitrios]
          • Create a server/daemon installation howto #5445 [Joel]
          • Add symlinks for the new daemon names #5131 [Joel]
          • Prepare a clear list of changes which need to be made to use new daemon names #5132 [Joel]
          • Create developers testing guide in the documentation #5452 [Joel, Mayank]
          • Exchange of function-based indices with normal indices #5440 [Mario, Martin, Radu]
          • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
          • Reduce rule tickets to les than 20 [Martin]
          • foreign key error on deleting dids in reaper #5733 [Martin]
          • Rules on containers in state OK but not all the files from the containers have locks #5447 [Martin]
          • Reduce logging of server & daemons #5449 [Mayank]
          • Add python 3.10 tests to CI framework. #5145 [Mayank]
          • Release webUI 2.0 beta version [Mayank]
          • rework prometheus/statsd metric names #5804 [Radu]
          • Re-work/cleanup distance table #5454 [Radu]
          • rework conveyor-throttler #5805 [Radu]
          • Switch to pytest [Radu?]
            • Needs discussion
          • Create documentation page for S3-type storage #5450 [Rizart]
          • Add support for S3 virtual hosted buckets #5451 [Rizart]
            • Test on-premise
            • Ongoing support issue with AWS/Fresno X.509 shenanigans
          • Merge list_dids and list_dids_extended methods #5448 [Rob]
        • Delayed
      • Other topics
    • 5
      AOB