Rucio Meeting

Thursday 14 Sept 2023, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• DIRAC & Rucio workshop - registration deadline on monday!
  • 41 participants so far

15:05 → 15:25 Community News & DevOps roundtable

• ATLAS
  • 32.3.0 deployed
  • Testing for using --deep for all list_dataset_replica requests
    • For now looks promising (Last test had unrelated problems)
• CMS
• Fermilab DUNE/ICARUS/RUBIN
  • Rucio 32 deployment running in development
    • Everything but WebUI and monitoring
    • Name switch for hermes <-> hermes_legacy
  • Q: statsd / prometheus / both?
    • Expose the same metrics
    • Any intention of removing statsd? -> no
  • Q: automatix client auth part not working
    • helm-charts updated with 0600 permissions for the x509_up file
    • Still same problem with auth
    • 1.29.6 vs 1.29.12 -> should not matter
    • Brandon: Fixing volume for etc/grid-security certificates
  • RUBIN: Data management workshop 
    • Data archival on rucio-backed tape RSE
    • Custom non-det algorithm to store PFN
    • Archive-ZIP file features in Rucio (Martin will send link to Brandon)
• Belle II
• ALTO
  • Network system class @ Yale will cover Rucio!
• KM3Net
  • Trying to setup rucio for KM3Net
  • Setting up RSEs
  • Struggling with database a bit
    • MariaDB vs. Postgres
    • Userlogin to Rucio stopped working with switch to Postgres?

15:25 → 15:45 WebUI 2.0 updates

Speakers: Mayank Sharma (University of Michigan (US)), Anton Schwarz (Heidelberg University (DE))

Rucio_WebUI 2.0_Rucio_Developers Meeting_14_Sept_2023_v1.pdf

Rucio_WebUI 2.0_Rucio_Developers Meeting_14_Sept_2023_v1.pptx

15:45 → 15:55 Developers roundtable

Progress-2023-09-14.png

• Rucio 33 "Eternal Sunshine of the Donkey's Mind" roadmap
  • In Progress
    • Design and implement a token cache module #6183 [Dimitrios]
    • Introduce the RSE decommissioner daemon #6321 [Yutaro, Dimitrios]
    • WebUI Release 33 page tracker #301 [Mayank/Anton]
      • Added list_file_replicas and list_dataset_replicas functionality added
  • In Review
    • Unable to Delete File DID via Undertaker #5154 [Anton]
    • Incorporate IAM-based testing for token workflows #6188 [Eraldo, Dimitrios]
  • Todo
    • Design and implement a token request module #6187 [Dimitrios]
    • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
    • foreign key error on deleting dids in reaper #5733 [Martin]
    • Review state of the Light Reaper #6317
    • Review lost data handling
    • Tests cleanup #6212
    • Transfer status visual representation #6189 [Radu]
    • Dynamic distance between RSEs #5012 [Radu]
    • revisit submitter source selection strategy #5776 [Radu]
  • Done
• Documentation corner
  • Documentation and dev guidelines for Mypy type annotations #116 [Mayank, Martin]
  • Document environmental variables affecting the client #171 [Dimitrios]
  • Improve documentation on rucio.cfg vs configuration table #183 [Radu]
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Add instruction about DB partitioning #185 [Martin]
  • bridge the gap between running rucio in demo env and full production deployment #187 [Radu]
  • Introduce documentation on subscriptions #190 [Cedric]
  • Page about the different event types and messages (ActiveMQ) #226 [Eraldo]
•  

15:55 → 16:00 AOB