Rucio Development Meeting

Thursday 4 Jul 2019, 15:00 → 16:30 Europe/Zurich

4/S-056 (CERN)

Martin Barisits (CERN)

15:00 → 15:10 News

• ESCAPE Workshop
• Meeting schedule for summer (July/August)
  • Bi-Weekly for July/August

15:10 → 15:30 News from the experiments

• ATLAS
  • Deletion issue
    • Due to archive workflow
    • Cedric made a fix for this which will go into next weeks release
• Multi-VO/Andrew/Ian
  • rse_id handling with core returning names as well
  • PR coming next week for this
  • Probes which use core functions might be impacted
    • We will have a look on their repo
• DUNE/James
  • Performance tests on URL signed code
  • CHEP publication on event service
  • S3 signing code fixing for AWS

15:30 → 15:50 Hot topics

15:50 → 16:20 Developers roundtable

• Rucio 1.21 priority followup
  • Focus
    • Cleanup & Stability
    • Documentation
    • Deployment (Kubernetes!)
  • Open ID connect #2612
    • Will use refresh tokens to store in the database
    • Need to setup auth server for testing
    • --> Possibility to test with a pre release for ESCAPE?
    • Lifetime of refresh token is infinite at the moment
  • Rucio.cfg vs Rucio config table cleanup #2630
    • Will start a document and have every component responsible to comment;
  • Documentation for configuration parameters #2631
  • History table definition explicitly (No Versioned models) #2063
  • MultiVO features #2635
  • Reaper 2.0 #2412
    • PR for the first prototype is there
      • Already being tested on small amount sites
    • Relax protection of sources #1637
      • More complicated than we thought;
      • Query should scale but need to see if it will hold under heavy load;
    • Fill contents history when deleting last replica #37
    • Archive Handling in Reaper #1431
      • Current issue we discussed in Community news
    • Reaper protection in case of Judge-Evaluator backlog #1578
  • Operators documentation / recipes #2636
  • Expand Kubernetes usage
    • conveyor-submitter, poller, finisher and undertaker are now running
      • Running stable now
  • Tracking what happened with a did #2637
    • In progress; Some code from the coding camp, but needs to be continued
  • XCache config table population add to probe #2638
    • Needs to be commited
  • BB8 Needs better configuration and get rid of hard-coded entries
  • Better way to deal with configuration/permissions (entry point, configuration.py, …) #533
  • Transparent handling of archives with rules #1091
  • Global Quotas #2315
  • Possibility to inject rules delayed #2639
  • Improve oracle test crashes #2588
    • Some improvements, unclear if it helped
  • Python 3.5 for server
    • pystatsd does not support Python3
      • Change to statsd possible
    • Postgres issue with byte and text data
  • Python 3.6 for clients
    • Should be compatible, but something missing in the tests
  • Changes for CTA transfer handling #2632
    • Non trivial, still needs to be discussed
    • Could do multi-hop handling right away with this, but should focus development on CTA handling first
  • Source throttling #2611
    • Almost done; Need to add src_rse_id
    • Now using rse_transfer limits table
    • Will need schema change as well
  • Activity exclusion for submitter #2640
• ​​​​​​​​​​​​​​Verify performance of highly used commands (list_replicas)
  • Probe which randomly tests performance of these commands

16:20 → 16:30 AOB

• Next meeting in 2 weeks!