Rucio Meeting

Thursday 20 Mar 2025, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

1 News

• 37 "Dungeons & Donkeys"
  • Code Freeze on Mar-20 08:00 CET
  • Mar-24 37.0.0rc1
  • Mar-31 37.0.0 FINAL
• Job vacancy @ CERN

2 Community News & DevOps roundtable

• ATLAS
  • Deployment of policy package
    • Works well, and is the recommended way to deploy them
    • Any other communities who want to try it?
    • Documentation will come asap
    • A second way of deploying them will be added as well
    • Target: 37.0.0 helm-charts to support this
• CMS
  • archive_metadata now in production
• Fermilab DUNE/RUBIN/*
  • RUBIN
    • Storage for raw data, distances, protocols, etc.
    • Precautions to not ever delete there
  • Fermilab CiLogon tokens
  • All Fermilab Rucio on 36.2
• DUNE
  • Download tokens
    • Not in 37.0, but should not slip by much
• HEPCCE
  • Trying to get Globus transfers to work
• ePIC
  • Figuring out multihop to tape
• Belle II
  • Moved to 35.6
  • Will move to 37 later
• ESCAPE
  • Open Data handling
  • ATLAS opendata on ESCAPE datalake
• CERN IT
  • Setup clusters for CERN SMEs
• DUNE Edinburgh
  • Tokens CiLogon works with 37.0 for general auth
  • For TPC it seems to work, but tests are ongoing
  • Schema development on policy package
    • policy package diff
• IHEP
  • Upgrading Rucio instance for HERD (36.*)

3 Developers roundtable

• Reflection about ongoing Sprint 4 (Sprint Board) [Maggie]
  • Seems good
• Priority Followup for Rucio 37 "Dungeons and Donkeys" major release
  • Todo
  • In Progress
    • Maintain backward compatibility with url's from old webui #131 [Mayank]
      • not a lot left
    • Rucio CLI >=36.0.0 Bugs and Suggestions #7224 [Maggie]
    • Fix static code analysis behaviour for functions decorated with @transactional_session, @read_session and @stream_session #6986[Riccardo]
      • Migration of other endpoints during 37 release lines
  • In Review
    • Include docker image publication in CI/CD #35 [Enrique]
      • Need to follow up
    • Rework OIDC user authentication #7310 [James, Anil]
    • New CLI: Consider switching away from argparse #7253 [Maggie] 
    • Upgrade to NextJS 15 and App Router #429 [Mayank]
  • Done
    • factorize duplicate messaging code into a common module or class #6423 [Alex]
    • psycopg2-binary should not be used in production - psycopg2 should be used instead #6669 [Dimitris] Sprint 1
    • Standardize permission policy check logic #7206 [James] Sprint 1
    • Build multi-arch images to support arm64 machines #354 [Riccardo] Sprint 1
  • Delayed
    • Create a new API endpoint for the clients to request tokens #6638 [Dimitrios]
      • Probably sooner, than later
    • Implement new token authentication for download #7029 [Dimitrios]
      • Needs more work
    • Improve handling of errors in Rucio REST API calls #39 [Enrique]
    • Integrate OIDC endpoints for Authorization Code Grant Type #507 [Mayank]
    • Rucio dependencies in test environments should be installed the same way ... #344 [Riccardo]
    • Testing: Run rucio/rucio integ tests for PRs in rucio/containers #310 [Riccardo]
    • Prevent multiple Reaper threads from working on the same replicas #6512[Hugo]
      • Need to incorporate reviews
• Miscellaneous
  • Setting up a Rucio development environment [Mayank]:
    • Some package related to PostgreSQL (psycopg2) causing issues. Applied a temporary workaround.
    • [Dimitris] working on a script to help boostrap the environment properly.
  • [Janusz] asking about failing tests in #Support
    • Exclude them in the (developers) test environment
    • PR is there
• (Luis) Discuss scope extraction on client side: default method. Should we rewrite it to simply split on the :? Currently it strips trailing / which can be an issue. It also has some user / group logic for historical reasons, can this also be removed? PR: https://github.com/rucio/rucio/pull/7521 https://github.com/rucio/rucio/blob/78ad9d452cdf73e4fff309a3fc086d7b05f33a79/lib/rucio/common/utils.py#L596
• (Luis) Discuss unifying how scope and name is extracted in server side. Currently using a regex, while in client side a method is used. I propose to unify this and use a method for all. https://github.com/rucio/rucio/issues/7519
• (Luis) Enable support for `/` encoded apache - https://github.com/rucio/containers/issues/387 - Any reason not to?
• Need to be careful with these changes in terms of backwards compatibility
• Community specific extract_scope method needs to be taken into account
• (Martin) https://hub.docker.com/repository/docker/rucio/apache_exporter/general
  • Is being used, location of dockerfile a mystery

4 AOB