Name: Rucio Meeting
Start: 2022-08-18T15:00:00+02:00
End: 2022-08-18T16:00:00+02:00
Location: No location set

15:00 → 15:05

News 5m

September meeting schedule
- Summer is over!
  - Back to weekly meetings
- Sep 1
- ~~Sep 8~~ Jeûne genevois
- Sep 15
- Sep 22
- Sep 29
Component lead update
- Core & Internals
  - Gabriele Fronzé stepped down
- Authentication & Authorisation
  - Rizart Dona stepped down
- Deletion
  - Radu Carpa added
- More updates to follow in the coming weeks

15:05 → 15:20

Community News & DevOps roundtable 15m

ATLAS
- Subject DNs in Legacy format
  - Need to figure out how to deal with the RFC-type DNs vs. Legacy DNs
  - IAM-X509 exports only RFC-type DNs
CMS
- 1.29 upgrade
  - Hoped it would fix a problem with large backlog
    - Transfer to tape with archive feature turned on
      - Initial failure was that these timed out
    - Then Rucio submits multi-source with file already exists error
    - Then on next re-submission where rucio should check existing checksums/sizes it does not work either
  - Further try:
    - Set all requests to FAILED for Rucio to resubmit
      - Did not help either
  - Need to clarify if FTS is reporting the correct checksum/data
    - Fermilab FTS is being used, but should be the same as CERN
    - Radu will provide some pointers
- Issue with DATASETS/CONTAINERS added to CONTAINERS do not follow the ALL grouping
  - Logic was never implemented
  - Will focus on the DATASET attached to CONTAINER case first, more complicated CONTAINER to CONTAINER one later
Belle II
- Upgraded to 1.28.5 for Server and 1.28.7 for daemons
  - Seems to work fine
- Activated temporary tables as well
  - Should only be the reaper for now; submitter as well if you run it on limited number of RSEs
  - For the rest need to go to 1.29, but need Py3 first
  - Need to check utilisation plots of database
DUNE
- Tape backed RSEs
  - Use surl_algorithm for data placement
- Preparing for second part of data challenge
  - Planned for September
  - Process data --> Pilots will do rucio uploads
  - More Rucio interactions
RAL MutliVO
- Getting Authentication to work on K8s

15:20 → 15:25

Hot topics 5m

15:25 → 15:55

Developers roundtable 30m

Rucio 1.30 "The Donkeynator" progress tracking
- In Progress
  - Introduce heartbeat handler to all daemons #5443 [Cedric, Dimitrios, Mario]
    - Missing daemons: hermes, atropos, replicarecoverer
      - Hermes done
      - replicarecoverer Christoph will do it, Cedric will double check
      - atropos Dimitrios will check asap
  - Further ingetragtion of CMS-style consistency checking #5438 [Igor]
    - PR for REST interface needs merging
      - --> 1.29.3
    - From Rucio core PoV things are DONE
    - Igor will test daemonless consistency checking then on CMS instance
      - Once done, next steps need to be discussed
        General purpose utilities probably go to Rucio directly
        Other things probably to a separate repository for others to use
  - Adherence to rule grouping dependent on order of ATTACH #5251 [Igor]
  - Unable to Delete File DID via Undertaker #5154 [Martin]
    - More complicated than initially thought; still under investigation
- In Review
- Done
  - Remove old readthedocs documentation once the new one is fully operational #5446 [Joel, Martin]
- Todo
  - Refactor the different test workflows to a common one [Ben, Mayank]
  - Create a contributor guide for probes #84 [Dimitrios]
  - Create a server/daemon installation howto #5445 [Joel]
  - Add symlinks for the new daemon names #5131 [Joel]
  - Prepare a clear list of changes which need to be made to use new daemon names #5132 [Joel]
  - Create developers testing guide in the documentation #5452 [Joel, Mayank]
  - Exchange of function-based indices with normal indices #5440 [Mario, Martin, Radu]
  - Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
  - Reduce rule tickets to les than 20 [Martin]
  - foreign key error on deleting dids in reaper #5733 [Martin]
  - Rules on containers in state OK but not all the files from the containers have locks #5447 [Martin]
  - Reduce logging of server & daemons #5449 [Mayank]
  - Add python 3.10 tests to CI framework. #5145 [Mayank]
  - Release webUI 2.0 beta version [Mayank]
  - rework prometheus/statsd metric names #5804 [Radu]
  - Re-work/cleanup distance table #5454 [Radu]
  - rework conveyor-throttler #5805 [Radu]
  - Switch to pytest [Radu?]
    - Needs discussion
  - Create documentation page for S3-type storage #5450 [Rizart]
    - Exists as GGDoc (#5284)
  - Add support for S3 virtual hosted buckets #5451 [Rizart]
    - Test on-premise
    - Ongoing support issue with AWS/Fresno X.509 shenanigans
  - Merge list_dids and list_dids_extended methods #5448 [Rob]
- Delayed
Other topics

15:55 → 16:00

AOB 5m