Rucio Development Meeting

Martin Barisits (CERN)
Videoconference Rooms
Rucio Development Meeting
Zoom Meeting ID
Martin Barisits
Alternative host
Mario Lassnig
Useful links
Join via phone
Zoom URL
    • 15:00 15:10
      News 10m
      • Hermes2 evolution
        • Discussion scheduled for today but due to meeting conflicts, postponed to next week
      • Rucio CHEP Slides
        • @authors: Please add your slides
      • Rucio Community Workshop
        • Program committee had first meeting
        • First announcement will be sent out soon
      • 1.25.3.post1 hotfix release
        • Due to regression in hermes/messaging
    • 15:10 15:20
      Community News & DevOps roundtable 10m
      • ATLAS
        • More ATLAS K8s stuff
      • CMS
        • Setup integration instance for testing
          • On 1.25 now
      • LDMX
      • DUNE/Edinburgh
        • More LFN2PFN work
          • Integrating with the DUNE policy package
        • Lightweight client
          • Make the client run without config file, but run only with ENV variables
            • Yes, general interest in this
          • Will also have CERN Summer student to work on this
      • MultiVO/STFC
        • New RSEs integrated
          • Some issues to be fixed
      • ESCAPE
        • Move to flux for K8s config management
      • SKA
        • Stabilising the deployment
        • CRON job which dumps db to be able to restore if needed
        • Re-Configuring some of the RSEs
        • Data ingestion tests
        • Next: Probes
          • Are they commonly usable or still ATLAS specific?
            • Generally usable, still a few ATLAS specific ones (If you see ATLAS mentioned...)
            • Splitting the probes into common and community specific ones is still ongoing, but should be finished soon
          • Can you run them on K8s?
            • Yes
    • 15:20 15:40
      Hermes2 evolution (Postponed to next week) 20m
    • 15:40 15:55
      Developers roundtable 15m

      Rucio 1.26 "Donkey League of La Mancha" priority followup

      • In Progress
        • Test and switch to new rule algorithm #4215 [Martin]
          • Remove old code for 1.26
        • Migrate documentation to new docusaurus [Martin, Ben, Daniel]
          • PR is ready to remove doc/ folder
          • Need to add it then to documentation repo
          • Some missing changes
            • Release notes
            • Some other minor changes
          • Deprecation of readthedocs
            • Maybe add a header about the new documentation but leave the old documentation intact
            • Freezing it, but leaving it until full deprecation
        • Logging Review #4220 [All Component leads]
        • Re-Factor Transfer workflow [Radu]
          • Re-Structure PR merged
          • Now reviewing the bigger feature PR
        • Discuss and plan mysql5 deprecation [Mario]
          • No news from community
          • Remove support with 1.26, re-structure DB calls/migrate script with 1.27
          • Need to remove tests for mysql5 and do a historic cleanup of alembic
        • Test SSO Login [Rizart]
          • Studied API and look into connection to CERN SSO
        • Test Import and Export of CRIC JSON [Rizart]
          • ESCAPE needs custom parameters to add to RSEs 
            • Backported feature from ATLAS plugin to mainstream CRIC
            • Some fields in the RSE table, some attributes
            • Discuss with CRIC people to have a more consolidated solution
            • Checked how the different plugins are differentiated
              • Will possibly need a plugin for each community
          • ESCAPE already uses the rucio importer (full json import)
        • Auditor overhaul #3437 [Dimitrios, Eric] 
        • Hermes2 planning
          • Postponed to next week
        • Remove webpy endpoints and dependency #4044 [Ben]
          • Needs container repo PR first, to switch dev container to FLASK
      • Todo
        • Switch to policy-packages as default [James, Martin]
        • Versioned history tables #2063 [Martin]
        • Rename reaper2 to reaper #4462 [Martin]
        • rucio.cfg vs Config table #2630 [Mario] 
        • Python 3.8 and 3.9 testing #4453 [Ben]
        • and requirements overhaul #4456 [Ben]
        • Cleanup tickets #4463 [*]
      • Done
      • Delayed
        • QoS #3419 [Mario, Martin, Rizart]

      Developer roundtable

      • Integration testing
        • Issue with FTS transfers has been fixed 
          • PR is submitted for that
            • Using python2 to activate RSEs, delegate proxy
          • PR on containers repo to setup pip2 for fts-rest
        • TPC PR
          • Use fts rest CLI to validate actions of Rucio daemons
          • Need to move it to a GH action now
      • Issue with Daemons using the wrong DN
        • Lookup table to map VO to DNs
    • 15:55 16:00
      AOB 5m