Rucio Development Meeting

4/S-056 (CERN)



Show room on map
Martin Barisits (CERN)
    • 15:00 15:10
      News 10m
      • Rucio 1.21.0 "Donkeys of the Galaxy" Code Freeze on Sunday 23:59 CEST
        • Every PR not submitted by then will not be included in the feature release
    • 15:10 15:30
      News from the experiments 20m
      • ESCAPE
        • Escape CRIC instance with real escape data
        • Start probes via nagios or not?
          • Probably not needed to just run probe
        • Change rse_expression_parser test to initialize RSEs in the setup
      • ATLAS
        • -
      • CMS
        • Generalize probes (schema) so it is usable by other communities
        • Bring probes to K8s
          • Thomas: Some probes can be run as crons, others we should maybe change to report stats to prometheus
          • CMS has framework in place to send stuff to prometheus already
          • Probes which report table sizes (messages, etc.) to statsd (graphite) could just send it to prometheus instead
          • Possibly should also change the internal measurements to go to prometheus instead of graphite
        • Traces & Popularity system
          • Meeting after CHEP about this
      • Multi-VO
        • Move Andrews open PR to next feature release
    • 15:30 15:50
      Hot topics 20m
    • 15:50 16:20
      Developers roundtable 30m
      • Rucio 1.21 priority followup
        • Focus
          • Cleanup & Stability
          • Documentation
          • Deployment (Kubernetes!)
        • Open ID connect #2612
          • Provisioning client used similar to query VOMS (--> probes)
          • Will need a release candidate to test with e.g. ESCAPE
        • Rucio.cfg vs Rucio config table cleanup #2630
          • Will start a document and have every component responsible to comment;
          • Move to next one
        • Documentation for configuration parameters #2631
        • History table definition explicitly (No Versioned models) #2063
          • PR prepared
        • MultiVO features #2635
          • Open PR from Andrew, moved to next feature
        • Reaper 2.0 #2412
          • Needs improvement in the query to get list of unlocked replicas
            • SKIP LOCKED does not work due to being used as a subquery
            • Needs some more work with the query
          • Source protection: Implemented
          • Some minor other things might need improvement too
          • ATLAS running all Tier-3s, selected Tier-2s and one Tier-1 (RAL)
            • 60 threads (partly on K8s)
        • Operators documentation / recipes #2636
        • Expand Kubernetes usage
          • ATLAS: By the end of the month run all daemons/server on integration cluster
            • DONE
          • Next year: Move all production services to production as well
            • Depends on experience with integration service now
          • K8s receives 1-2% of the load now
            • Performance of http servers still a question
            • Increase load (5-10%) on ATLAS cluster in K8s
          • Using CERN IT infrastructure to collect logs to Hadoop and ES
        • Tracking what happened with a did #2637
          • Hannes submitted a PR, needs review
        • XCache config table population add to probe #2638
          • Needs to be commited
        • BB8 Needs better configuration and get rid of hard-coded entries
        • Better way to deal with configuration/permissions (entry point,, …) #533
          • Configuration comes from an external python package instead of Rucio core
          • Python package needs to have a module for schema, permission, policy
        • Transparent handling of archives with rules #1091
          • Move to next Feature release
        • Global Quotas #2315
          • Making progress; Updating the CLI
          • PR there, needs review
        • Possibility to inject rules delayed #2639
        • Improve oracle test crashes #2588
          • Some improvements, unclear if it helped
        • Python 3.5 for server
          • pystatsd does not support Python3
            • Change to statsd possible
          • fts3-rest-api removed, myproxy dependency removed as well;
          • Postgres issue with byte and text data
        • Python 3.6 for clients
          • Should be compatible, but something missing in the tests
        • Changes for CTA transfer handling #2632
          • Cedric and Martin will work on this next week
          • Move dev-container fts&xroot development over to travis container as well
          • Next test with CTA on their test instance
        • Source throttling #2611
          • Needs merging
        • Activity exclusion for submitter #2640
      • copy_pin_lifetime - Mario will work on it
      • staging times
        • JJ prepared PR for this
        • Also needs to be populated by conveyor poller
    • 16:20 16:30
      AOB 10m
      • Next week no meeting!
      • Nov 14: Rucio 1.22 release roadmap discussion