Rucio Development Meeting

Europe/Zurich
Martin Barisits (CERN)
Videoconference Rooms
Rucio Development Meeting
Zoom Meeting ID
413496641
Host
Martin Barisits
Alternative host
Mario Lassnig
Passcode
28849311
Useful links
Join via phone
Zoom URL
    • 15:00 15:10
      News 10m
      • Component lead changes due to recent and upcoming departures
        • Clients: Mario, Tomas, Radu
        • Consistency: Tomas, Dimitrios
        • Docker: Thomas, Mario, Radu
        • Monitoring: Thomas, Mario, Radu
        • Protocols: Mario, Tomas, Radu
        • Rebalancing: Tomas, Cedric
        • Recovery: Cedric, Tomas
        • REST & API: Ben, Martin
        • Traces: Thomas, Mario
        • Transfers: Martin, Cedric, Eric, Radu
        • WebUI: Thomas, Martin, Cedric​​​​​​​
      • ​​​​​​​Next week: Hermes 2 planning & discussion
    • 15:10 15:20
      Community News & DevOps roundtable 10m
      • ATLAS
        • ActiveMQ changes need to be applied
        • Issue with poller on Kubernetes and memory limit
          • Large amount of Staging requests which are very large, went slightly over memory limit
            • Increasing limit solved it
      • Belle II
        • Finally got WebUI running
          • Issue was that the WebUI did not have access to the DB
            • Needs access to check token
            • Error was misleading which lead to this difficult debugging
            • Another good reason to get rid of webui accessing the DB (Just redirecting)
              • In the meanwhile, better error message would already help
          • Some views are empty which rely on dumps
            • No real documentation for that
            • Subscription table (Showing all subscriptions) relies on a dump
              • Better move to REST endpoints and caching?
        • Metadata
          • Testing import of all Belle II metadata into Rucio (80M files, 10-15 fields per file)
      • LDMX
        • No news
      • DUNE/Edinburgh
        • LFN2PFN converter for DUNE
        • More lightweight client work soon
      • Multi-VO/RAL
        • Setting up some RSEs on the RAL instance
      • ESCAPE
        • No News
      • SKA
        • No News
    • 15:20 15:30
      Hot topics 10m
    • 15:30 15:55
      Developers roundtable 25m

      Rucio 1.26 "Donkey League of La Mancha" priority followup

      • In Progress
        • Test and switch to new rule algorithm #4215 [Martin]
          • Remove old code for 1.26
        • Migrate documentation to new docusaurus [Martin, Ben, Daniel]
          • man page generation for the doc
            • A bit different compared to sphinx
          • Documentation repo branch setup and running
          • Branch ready for rucio main repo
            • Can be merged soon
        • Logging Review #4220 [All Component leads]
        • Re-Factor Transfer workflow [Radu]
          • Re-Structure PR merged
          • Now reviewing the bigger feature PR 
        • Discuss and plan mysql5 deprecation [Mario]
          • No news from community
          • Remove support with 1.26, re-structure DB calls/migrate script with 1.27
        • Test SSO Login [Rizart]
          • Studied API and look into connection to CERN SSO
        • Test Import and Export of CRIC JSON [Rizart]
          • ESCAPE needs custom parameters to add to RSEs 
            • Backported feature from ATLAS plugin to mainstream CRIC
            • Some fields in the RSE table, some attributes
            • Discuss with CRIC people to have a more consolidated solution
        • Auditor overhaul #3437 [Dimitrios, Eric] 
        • Hermes2 planning
      • Todo
        • Switch to policy-packages as default [James, Martin]
        • Versioned history tables #2063 [Martin]
        • Rename reaper2 to reaper #4462 [Martin]
        • rucio.cfg vs Config table #2630 [Mario] 
        • Python 3.8 and 3.9 testing #4453 [Ben]
        • Remove webpy endpoints and dependency #4044 [Ben]
        • Setup.py and requirements overhaul #4456 [Ben]
        • Cleanup tickets #4463 [*]
      • Done
      • Delayed
        • QoS #3419 [Mario, Martin, Rizart]

      Developer roundtable

      • Integration testing
        • FTS container not doing transfers, needs some work
      • Testing issue with Deadlocks
        • Radu made 2 PRs to fix some of these
    • 15:55 16:00
      AOB 5m