Rucio Development Meeting

Europe/Zurich
304/1-001 (CERN)

304/1-001

CERN

20
Show room on map
Martin Barisits (CERN)
Videoconference Rooms
Rucio_Development_Meeting
Name
Rucio_Development_Meeting
Description
Rucio Development Meeting
Extension
10760955
Owner
Martin Barisits
Auto-join URL
Useful links
Phone numbers
    • 15:00 15:10
      News 10m
      • Release schedule
        • 1.20.5 on Sep 6
        • 1.20.6 on Sep 23
        • 1.20.7 on Oct 14
        • (Oct 15 - Oct 17 Rucio coding camp)
        • 1.21 "Donkeys of the Galaxy" end October/begin November
      • Rucio journal paper released
      • Meeting schedule
        • September we resume the weekly development meeting schedule
        • Sep 5 NO MEETING (CERN holiday)
    • 15:10 15:30
      News from the experiments 20m
      • ATLAS
        • Summer holidays!
      • CMS
        • Rucio data export
        • Distance export issue
          • Will be tracked down
        • Release candidate build of docker containers does not work - builds the latest non RC version
      • DUNE/Edinburgh
        • Two open PRs: Object stores and Policy/Permissions
      • ESCAPE
        • Import/Export
          • accounts, quotas did not work
        • Generic metadata
          • Benchmark on postgres and mysql
            • postgres for 10mio rows finding a row took 2s
            • mysql takes ~2m
            • Oracle tests will come soon
          • Undertaker bug found
            • Different schema with dev container than production version?
            • Need to identify this
    • 15:30 15:50
      Hot topics 20m
    • 15:50 16:20
      Developers roundtable 30m
      • Rucio 1.21 priority followup
        • Focus
          • Cleanup & Stability
          • Documentation
          • Deployment (Kubernetes!)
        • Open ID connect #2612
          • Provisioning client used similar to query VOMS (--> probes)
          • Will need a release candidate to test with e.g. ESCAPE
        • Rucio.cfg vs Rucio config table cleanup #2630
          • Will start a document and have every component responsible to comment;
        • Documentation for configuration parameters #2631
        • History table definition explicitly (No Versioned models) #2063
        • MultiVO features #2635
        • Reaper 2.0 #2412
          • PR for the first prototype is there
            • Already being tested on small amount sites
          • Relax protection of sources #1637
            • More complicated than we thought;
            • Query should scale but need to see if it will hold under heavy load;
          • Fill contents history when deleting last replica #37
          • Archive Handling in Reaper #1431
            • Current issue we discussed in Community news
          • Reaper protection in case of Judge-Evaluator backlog #1578
        • Operators documentation / recipes #2636
        • Expand Kubernetes usage
          • conveyor-submitter, poller, finisher and undertaker are now running
            • Running stable now
            • Almost all daemons are running now on kubernetes
            • hermes and reaper
          • All daemons running now on Kubernetes
          • Should add servers
          • Plan is to move from ATLAS integration puppet to kubernetes
        • Tracking what happened with a did #2637
          • In progress; Some code from the coding camp, but needs to be continued
        • XCache config table population add to probe #2638
          • Needs to be commited
        • BB8 Needs better configuration and get rid of hard-coded entries
        • Better way to deal with configuration/permissions (entry point, configuration.py, …) #533
        • Transparent handling of archives with rules #1091
        • Global Quotas #2315
          • Making progress; Updating the CLI
          • PR there, needs review
        • Possibility to inject rules delayed #2639
        • Improve oracle test crashes #2588
          • Some improvements, unclear if it helped
        • Python 3.5 for server
          • pystatsd does not support Python3
            • Change to statsd possible
          • Postgres issue with byte and text data
        • Python 3.6 for clients
          • Should be compatible, but something missing in the tests
        • Changes for CTA transfer handling #2632
          • Non trivial, still needs to be discussed
          • Could do multi-hop handling right away with this, but should focus development on CTA handling first
        • Source throttling #2611
          • Almost done; Need to add src_rse_id
          • Now using rse_transfer limits table
          • Will need schema change as well
          • One test failing because of sqlalchemy with sqlite; Otherwise development is done
        • Activity exclusion for submitter #2640​​​​​​​
    • 16:20 16:30
      AOB 10m
      • No meeting next week