• Elasticsearch
  • Some cluster stability issues. We anyhow need an upgrade so we will rebuild cluster next week.
• XCache
  • Added new node to MWT2. We will rebuild the one causing issues.  
  • Need a stronger xcache node at AGLT2. This one can't keep up: 
  • Added a SLATE installed xcache node to Wuppertal. Work on setting up their VP queue. This is a test for the larger deployment now that Germany is getting rid of ATLAS sites at Universities and moving resources to HPCs.
  • We need better ticketing when xcache has issue. If Squid people can monitor and ticket sites (and there are many more of them) we should be able to do that for XCaches too. We have enough tests, we just need another person to look at them every day and create a ticket if needed
• Varnishes
  • work fine but we get tickets when we move traffic from one to another:
  •