Rucio Development Meeting
-
- 15:00 → 15:10
-
15:10
→
15:30
News from the experiments 20m
- ESCAPE
- Escape CRIC instance with real escape data
- Start probes via nagios or not?
- Probably not needed to just run probe
- Change rse_expression_parser test to initialize RSEs in the setup
- ATLAS
- -
- CMS
- Generalize probes (schema) so it is usable by other communities
- Bring probes to K8s
- Thomas: Some probes can be run as crons, others we should maybe change to report stats to prometheus
- CMS has framework in place to send stuff to prometheus already
- Probes which report table sizes (messages, etc.) to statsd (graphite) could just send it to prometheus instead
- Possibly should also change the internal measurements to go to prometheus instead of graphite
- Traces & Popularity system
- Meeting after CHEP about this
- Multi-VO
- Move Andrews open PR to next feature release
- ESCAPE
-
15:30
→
15:50
Hot topics 20m
-
15:50
→
16:20
Developers roundtable 30m
- Rucio 1.21 priority followup
- Focus
- Cleanup & Stability
- Documentation
- Deployment (Kubernetes!)
- Open ID connect #2612
- Provisioning client used similar to query VOMS (--> probes)
- Will need a release candidate to test with e.g. ESCAPE
- Rucio.cfg vs Rucio config table cleanup #2630
- Will start a document and have every component responsible to comment;
- Move to next one
- Documentation for configuration parameters #2631
- History table definition explicitly (No Versioned models) #2063
- PR prepared
- MultiVO features #2635
- Open PR from Andrew, moved to next feature
- Reaper 2.0 #2412
- Needs improvement in the query to get list of unlocked replicas
- SKIP LOCKED does not work due to being used as a subquery
- Needs some more work with the query
- Source protection: Implemented
- Some minor other things might need improvement too
- ATLAS running all Tier-3s, selected Tier-2s and one Tier-1 (RAL)
- 60 threads (partly on K8s)
- Needs improvement in the query to get list of unlocked replicas
- Operators documentation / recipes #2636
- Expand Kubernetes usage
- ATLAS: By the end of the month run all daemons/server on integration cluster
- DONE
- Next year: Move all production services to production as well
- Depends on experience with integration service now
- K8s receives 1-2% of the load now
- Performance of http servers still a question
- Increase load (5-10%) on ATLAS cluster in K8s
- Using CERN IT infrastructure to collect logs to Hadoop and ES
- ATLAS: By the end of the month run all daemons/server on integration cluster
- Tracking what happened with a did #2637
- Hannes submitted a PR, needs review
XCache config table population add to probe #2638Needs to be commited
- BB8 Needs better configuration and get rid of hard-coded entries
- Better way to deal with configuration/permissions (entry point, configuration.py, …) #533
- Configuration comes from an external python package instead of Rucio core
- Python package needs to have a module for schema, permission, policy
Transparent handling of archives with rules #1091Move to next Feature release
- Global Quotas #2315
- Making progress; Updating the CLI
- PR there, needs review
- Possibility to inject rules delayed #2639
Improve oracle test crashes #2588Some improvements, unclear if it helped
- Python 3.5 for server
- pystatsd does not support Python3
- Change to statsd possible
- fts3-rest-api removed, myproxy dependency removed as well;
Postgres issue with byte and text data
- pystatsd does not support Python3
Python 3.6 for clientsShould be compatible, but something missing in the tests
- Changes for CTA transfer handling #2632
- Cedric and Martin will work on this next week
- Move dev-container fts&xroot development over to travis container as well
- Next test with CTA on their test instance
- Source throttling #2611
- Needs merging
Activity exclusion for submitter #2640
- Focus
- copy_pin_lifetime - Mario will work on it
- staging times
- JJ prepared PR for this
- Also needs to be populated by conveyor poller
- Rucio 1.21 priority followup
- 16:20 → 16:30