Rucio Development Meeting

Martin Barisits (CERN)
Rucio Development Meeting
Zoom Meeting ID
Martin Barisits
Alternative host
Mario Lassnig
Useful links
Join via phone
Zoom URL
    • 3:00 PM 3:05 PM
      News 5m
      • May meeting schedule
        • May-6
        • May-13 Holiday in Geneva
        • May-20 vCHEP parallel sessions
        • May-27
    • 3:05 PM 3:15 PM
      Community News & DevOps roundtable 10m
      • ATLAS
        • ATLAS Local Rootbase
          • Switch to Python3.7 for Rucio clients (Dual stack deployment, default for wrapper is py3, for non-wrapper deployment still py2)
            • Need discussion for Python2 client-support continuation
            • Ideally to discuss at the Rucio Community Workshop
      • CMS
        • Setup GO transfers for CMS
          • Trying to setup infrastructure
          • Ask Matt to extend the documentation for rse_settings for GO
      • Fermilab
        • Why grafana monitoring is not worning
          • No data ever shows up
      • Belle II
        • Metadata ingesting
        • Plenary talk at vCHEP for Belle Rucio integration
        • Working on webUI
          • Trying to setup the dumps
          • Dump only works on CEST, not other timezones --> Should be switched to UTC
      • Regensburg Lattice QCD
        • Waiting to setup new test env
      • LDMX
      • RAL / MultiVO
        • Transfers with MultiVO not working
        • All transfers being done under one certificate
          • Needs VO - Certificate mapping in conveyor
            • Possible issue with grouping method
      • ESCAPE
        • Infrastructure updates
        • Auth server + openid connect
          • Redirection back from IAM to auth server does not work
          • Possibly ingress controller on K8s preventing it
      • SKAO
        • Data ingestion tests
        • Policy packages
          • James doc page describing how to write/integrate it
        • GO
    • 3:15 PM 3:35 PM
      Hermes2 evolution 20m
      Speakers: Cedric Serfon (Brookhaven National Laboratory (US)) , Mario Lassnig (CERN) , Dr Riccardo Di Maria (CERN) , Rizart Dona (CERN) , Martin Barisits (CERN)
    • 3:35 PM 3:55 PM
      Developers roundtable 20m

      Rucio 1.26 "Donkey League of La Mancha" priority followup

      • In Progress
        • Test and switch to new rule algorithm #4215 [Martin]
          • Remove old code for 1.26
        • Migrate documentation to new docusaurus [Martin, Ben, Daniel]
          • PR is ready to remove doc/ folder
          • Need to add it then to documentation repo
          • Some missing changes
            • Release notes
            • Some other minor changes
          • Deprecation of readthedocs
            • Maybe add a header about the new documentation but leave the old documentation intact
            • Freezing it, but leaving it until full deprecation
        • Logging Review #4220 [All Component leads]
        • Re-Factor Transfer workflow [Radu]
          • Re-Structure PR merged
          • Now reviewing the bigger feature PR
        • Discuss and plan mysql5 deprecation [Mario]
          • No news from community
          • Remove support with 1.26, re-structure DB calls/migrate script with 1.27
          • Need to remove tests for mysql5 and do a historic cleanup of alembic
        • Test SSO Login [Rizart]
          • Studied API and look into connection to CERN SSO
        • Test Import and Export of CRIC JSON [Rizart]
          • ESCAPE needs custom parameters to add to RSEs 
            • Backported feature from ATLAS plugin to mainstream CRIC
            • Some fields in the RSE table, some attributes
            • Discuss with CRIC people to have a more consolidated solution
            • Checked how the different plugins are differentiated
              • Will possibly need a plugin for each community
          • ESCAPE already uses the rucio importer (full json import)
        • Auditor overhaul #3437 [Dimitrios, Eric] 
        • Hermes2 planning
          • Postponed to next week
        • Remove webpy endpoints and dependency #4044 [Ben]
          • Needs container repo PR first, to switch dev container to FLASK
      • Todo
        • Switch to policy-packages as default [James, Martin]
        • Versioned history tables #2063 [Martin]
        • Rename reaper2 to reaper #4462 [Martin]
        • rucio.cfg vs Config table #2630 [Mario] 
        • Python 3.8 and 3.9 testing #4453 [Ben]
        • and requirements overhaul #4456 [Ben]
        • Cleanup tickets #4463 [*]
      • Done
      • Delayed
        • QoS #3419 [Mario, Martin, Rizart]

      Developer roundtable

      • Integrating Belle DIRAC Rucio solution into Multi-VO RAL rucio server
        • Rucio clients are delivered with DIRAC workload
        • Goal is to use DIRAC Configuration to configure Rucio
        • Right now rucio clients strictly require a config file
          • $RUCIO_HOME can specify directory where rucio.cfg is located
        • Currently import rucio clients
          • logging is setup on input level
        • Not loading rucio.cfg but read config from ENV instead?
        • Issue is that during import of the client already setup_logger is called, which crashes without config
          • Followup offline
          • Problem might extend outside the setup_logger aswell
      • Integration testing
        • Change requests all implemented
        • Changed pytest plugin
        • Should work now
    • 3:55 PM 4:00 PM
      AOB 5m