Rucio Meeting

Thursday 18 Apr 2024, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

1 News

• Rucio 34.3.0 release on May 6 (in 3 weeks instead of 2)
• Migrating from Twitter to Mastodon
  • https://fosstodon.org/@rucio

2 Community News & DevOps roundtable

• ATLAS
• CMS
  • Upgraded to Rucio 34
    • scheme_list vs. protocol priority? 
      • RUBIN observing a similar issue
      • Issue # to be followed up on
    • downgraded to 33 after
  • RSE Decomissioner daemon question
    • How to start/deploy it:
      • Dimitrios: We need to provide a helm-chart for it first, but then it should be rather trivial
      • Andres can help creating the chart, Dimitrios can give pointers
• Fermilab
  • RUBIN
    • Automatix issue
      • Protocol mismatch still happening (Might be related to the scheme_list issue pointed out above)
  • DUNE
    • DB issues
    • Integration of multiple systems (metadata, workflow, rucio)
  • FNAL
    • Not a lot of news
• ESCAPE and co
• DUNE / Edinburgh
  • Troublshooting issues
  • VO tests PR
    • Mayank helping to align
• RAL / MultiVO
  • Running 34.2
    • Also having the issue from above
      • Transmogrifier impacted?
• CERN IT
  • Prototype deployment for COMPASS experiment

3 Developers roundtable

Progress-2024-04-18.png

• Rucio 35 "Donkey and the Data Factory" roadmap
  • In Progress
    • Review operations where SSL verification is disabled #6632 [Dimitrios]
      • Needs more testing (Change in container was needed beforehand)
      • Will resume next week
      • BDT discussion yesterday: Long term: /etc/grid-security/ will dissapear (Sites should use system trust)
      • Q: How to update the certificates (in the future)
        • Long term: hopefully wont necessary
        • NOW: Populate certificates in a volume externally, /etc/grid-security/ will be added to system trust
    • Finalize SQLAlchemy 1.0 -> 2.0 syntax migration #6640 [Erling]
    • Make a table for all daemon and its functionality #296 [Hugo]
    • overwrite_when_only_on_disk #6581 [Hugo]
    • Prepare replacement of current policy import with policy packages #4798 [James]
      • Need to move existing upstream policies
        • ATLAS
        • Belle II to be checked
        • CMS DONE
        • DOMA TPC to be checked
        • ESCAPE to be checked
        • Icecube to be checked
        • LSST / RUBIN DONE
    • Re-write and standardization of client command structure #6639 [Maggie]
    • Webui 35 tracker 1/34 [Mayank]
    • Type annotate the code #6588 [Riccardo]
  • In Review
    • Collect functional RSE attributes into a single file #6624 [Dimitrios, Mira]
    • Remove all instances of mutable objects (e.g. empty lists or dictionaries) as default parameter values #5169 [Erling]
    • JupyterLab version (v3 to v4) update #27 [Francesc, Enrique]
      • Debugging
    • Formatting / style guide #287 [Riccardo, Dimitrios, Maggie]
      • Main PR is merged
      • SQLAlchemy style guide

• • Todo
    • Unable to Delete File DID via Undertaker #5154 [Alex]
    • factorize duplicate messaging code into a common module or class #6423 [Alex]
    • SUBSCRIPTIONS_HISTORY integrity error #6292 [Cedric]
    • Make new token support Multi-VO compatible #6406 [Dimitrios]
    • Use OpenID Connect Discovery #6414 [Dimitrios]
    • Allow for token providers other than IAM #6630 [Dimitrios]
    • Make WebDAV protocols on par with GFAL #6633 [Dimitrios]
    • bridge the gap between running rucio in demo env and full production deployment #187 [Enrique, Hugo]
    • Retire RSE limits #6629 [Erling, Dimitrios]
    • Rename internal API layer to gateway #6644 [Erling]
    • Prevent multiple Reaper threads from working on the same replicas #6512 [Hugo]
    • Evaluate all TODO and FIXME comments #6642 [Martin]
    • Testing: Client Only tests #6599 [Mayank, Maggie]
    • Evaluate archive workflows for rules (Using an archive as a source) #6643 [Riccardo]
    • Improving docstrings #6628 [Riccardo]
    • Create and populate common monitoring repository #6641 [Tim]
  • Done
  • Delayed
• Documentation corner
  • Documentation and dev guidelines for Mypy type annotations #116 [Mayank, Martin]
  • Document environmental variables affecting the client #171 [Dimitrios]
  • Improve documentation on rucio.cfg vs configuration table #183 [Maggie]
    • In review
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Add instruction about DB partitioning #185 [Martin]
  • bridge the gap between running rucio in demo env and full production deployment #187 [Enrique, Hugo]
  • Introduce documentation on subscriptions #190 [Cedric]
  • Formatting / style guide #287 [Riccardo, Dimitrios, Maggie]
  • Document how deletion occurs. #288 [Anil]
  • Documentation for RSE settings #293 [???]
  • Make a table for all daemon and its functionality #296 [Hugo]
  • Testing instructions #309 [Maggie]
    • In review
• Other topics
  •  

4 AOB