Rucio Meeting

Thursday 1 Feb 2024, 15:00 → 16:00 Europe/Zurich

Dimitrios Christidis (CERN)

15:00 → 15:05 News

• February release schedule
  • Feb-05 33.?
  • Feb-12 1.29.17 LTS (Might be delayed to Mar-11)
  • Feb-12 32.? LTS (Might be delayed to Mar-11)
  • Feb-19 33.?
  • More releases as needed during DC24

15:05 → 15:25 Community News & DevOps roundtable

• ATLAS
  • Currently conducting a test with file-specific tokens for the transfer destinations
    • Have not identified any problems 
    • This will eventually be offered in the future for any communities who would like to run with such a configuration
  • Submitter a proposal for the CERN Summer Student Programme to improve the Rucio CLI using the ‘Rich’ library
  • Proposals for GSOC?
    • Deadline is on the 9th.
• CMS
  • Running 33.3 on production
  • Successful token transfers; number of sites is gradually increasing
  • Large-scale pre-DC24 test scheduled for Tuesday
  • Work to make the Suspicious Replica Recoverer VO-agnostic (#6396)
    • Awaiting for some answer
  • Keeping track of which account deleted a rule (#6360)
    • Can start the implementation with recording the account, but discussion needed on whether it is possible store the identity that was used to authenticate to said account.
• Fermilab / DUNE / RUBIN
  • Mu2e
    • Working on integrating Rucio with other systems (e.g. MetaCat)
    • Also interested in in-place ingest client
  • ICARUS
    • Work on in-place ingest client
  • DUNE
    • To test release 33 deployment, some issues identified in the client
      • Some apparent permission issue with upload
      • Occasionally, the creation of datasets fail
  • RUBIN
    • Work on making use of Automatix
    • Work on a policy package
• ESCAPE
  • NTR
• ALTO
  • Expecting to finish the testing for the Web UI monitoring next week

15:25 → 15:35 Token Q&A

Speaker: Dimitrios Christidis (CERN)

• Monitoring token transfers through Grafana at CERN (RQF2537926)
  
  • Now possible through https://monit-grafana.cern.ch/d/CIjJHKdGk/fts-transfers?orgId=20
• Support for token providers beyond IAM
  • CILogon is next. Rucio’s plan for a plugin-based architecture still stands. It will allow the implementation of different providers.
  • Will need some assistance to (1) verify that the tokens needed by Rucio can be acquired and (2) test the workflows to do it.

15:35 → 15:55 Developers roundtable

Progress-2024-02-01.png

• Rucio 34 "Donkey Potter and the Data Cache" roadmap
  • In Progress
    • foreign key error on deleting dids in reaper #5733 [Alex]
    • factorize duplicate messaging code into a common module or class #6423 [Alex]
    • Continue migration to SQLAlchemy 2.0 syntax #6057 [Erling]
    • Deployment and Release Workflow #401 [Mayank, Eraldo]
      • 33.ALPHA tagged
        • Will still need a reverse proxy
        • Helm charts being worked on
          • Progress made, but some difficulties with certificates
    • Missing WebUI Release 33 page tracker #301 [Mayank, Eraldo]
    • Unable to Delete File DID via Undertaker #5154 [Riccardo]
    • Type annotate the code #6454 [Riccardo]
    • Update extension for v32 (and higher) compatibility #25 [Francesc, Enrique]
      • Minor documentation issues being worked on
  • In Review
    • Refactor policy package algorithm code #6382 [James]
    • Metadata for tape co-location and transfer prority #6398 [Maggie]
    • Update/Re-design core.meta module #5224 [Maggie, Rob]
  • Todo
    
    • Migrate Dashboard to Clean Architecture #158 [Mayank, Eraldo]
    • bridge the gap between running rucio in demo env and full production deployment #187 [Radu, Enrique]
  • Done
    • Add Token based TPC tests to the CI #6451 [Radu]
  • Delayed
• Documentation corner
  • Documentation and dev guidelines for Mypy type annotations #116 [Mayank, Martin]
  • Document environmental variables affecting the client #171 [Dimitrios]
  • Improve documentation on rucio.cfg vs configuration table #183 [Radu]
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Add instruction about DB partitioning #185 [Martin]
  • bridge the gap between running rucio in demo env and full production deployment #187 [Radu]
  • Introduce documentation on subscriptions #190 [Cedric]
  • WebUI: Improve Docs #255 [Eraldo]
  • Add instructions for Mac Apple Silicon in the developer section #261 [Eraldo]
    • Under Review
  • Add Rucio QoS RSE description and instructions #268 [Matt]
    • Under Review
• Other topics
  •  

15:55 → 16:00 AOB