Rucio Development Meeting

Thursday 24 Oct 2019, 15:00 → 16:30 Europe/Zurich

4/S-056 (CERN)

Martin Barisits (CERN)

15:00 → 15:10 News

• Release schedule
  • 1.20.8 Oct 28
  • 1.21.0rc1 Nov 5
    • Code freeze Nov 3 23:59
  • 1.21.0 "Donkeys of the Galaxy" Nov 11
  • 1.21.1 Nov 25
• Dev meeting schedule
  • Nov 7 CANCELLED (CHEP)
  • Dec 5 CANCELLED (ATLAS S&C Week)
  • Dec 12 last Dev meeting in 2019
  • Jan 09 first Dev meeting in 2020
• Coding Camp 2019
  • Thanks for participating!
• Discussion Release schedule for 2020
  • Dev meeting Oct 31
• Rucio 1.22.0 "?" release roadmap planning
  • Dev meeting Nov 14

15:10 → 15:30 News from the experiments

• ESCAPE
  • Integration with CRIC
    • Importer configurable (master, etc)
      • RSE, Attribute, Protocol
    • Needs documentation
  • Import fails with big files
    • Aris needs access to one (Eric will provide)
• CMS
  • No news
• Multi-VO
  • Should try to get Andrews current PR into the next release
  • Eli will be back in the project in January
  • Database copy has not finished yet due to network issues
• DUNE/Edinburgh
  • No news
• ATLAS
  • Everything works 

15:30 → 15:50 Hot topics

Tape carousel (from Vincent):

• Pin lifetime from Rucio to FTS https://github.com/rucio/rucio/issues/3035
  • Will be implemented by Mario or Martin
• Suggested improvements about request history: https://github.com/rucio/rucio/issues/3023
  • JJ interested to do that
• Side ticket in gfal: https://its.cern.ch/jira/browse/DMC-1187

15:50 → 16:20 Developers roundtable

• Rucio 1.21 priority followup
  • Focus
    • Cleanup & Stability
    • Documentation
    • Deployment (Kubernetes!)
  • Open ID connect #2612
    • Provisioning client used similar to query VOMS (--> probes)
    • Will need a release candidate to test with e.g. ESCAPE
  • Rucio.cfg vs Rucio config table cleanup #2630
    • Will start a document and have every component responsible to comment;
  • Documentation for configuration parameters #2631
  • History table definition explicitly (No Versioned models) #2063
  • MultiVO features #2635
  • Reaper 2.0 #2412
    • Needs improvement in the query to get list of unlocked replicas
      • SKIP LOCKED does not work due to being used as a subquery
    • Source protection: Implemented
    • Some minor other things might need improvement too
  • Operators documentation / recipes #2636
  • Expand Kubernetes usage
    • ATLAS: By the end of the month run all daemons/server on integration cluster
    • Tomorrow: Meeting with CERN-IT to discuss service level of K8s at CERN
    • Next year: Move all production services to production as well
      • Depends on experience with integration service now
  • Tracking what happened with a did #2637
    • Hannes submitted a PR, needs review
  • XCache config table population add to probe #2638
    • Needs to be commited
  • BB8 Needs better configuration and get rid of hard-coded entries
  • Better way to deal with configuration/permissions (entry point, configuration.py, …) #533
    • Configuration comes from an external python package instead of Rucio core
    • Python package needs to have a module for schema, permission, policy
  • Transparent handling of archives with rules #1091
    • Move to next Feature release
  • Global Quotas #2315
    • Making progress; Updating the CLI
    • PR there, needs review
  • Possibility to inject rules delayed #2639
  • Improve oracle test crashes #2588
    • Some improvements, unclear if it helped
  • Python 3.5 for server
    • pystatsd does not support Python3
      • Change to statsd possible
    • Postgres issue with byte and text data
  • Python 3.6 for clients
    • Should be compatible, but something missing in the tests
  • Changes for CTA transfer handling #2632
    • Cedric and Martin will work on this next week!
  • Source throttling #2611
    • Needs merging
  • Activity exclusion for submitter #2640
• ​​​​​​​

16:20 → 16:30 AOB