Issues:

• ECHO problems (faulty disk on the 30th of April causing some slow IOPs + slow IOPs for uknown reason on the 7th of May) caused file loss and corruption (GGUS 683184)
  • Lost files restored, xrootd uploads disabled, to avoid race conditions
    • around 1k files lost, all deleted or restored now
  • Full list of corrupted files is still to be identified.
    • So far only one was found
    • Would it be possible to get a list of all files with their sizes and checksums?
• DIRAC issues on Tuesday (night + early morning)
  • Resuted in increase in completed and rescheduled jos
• Spikes of failed WGProduction jobs
  • Buggy xrootd client version used (5.3), can not be helped
    • This version is unable to execute any vector read request if it has more than two chunks in it
      • Therefore such jobs will not work at any site with Xrootd storage
• Failed uploads from HLTFarm
  • Expected, due to lack of network connectivity
• LHCb certification pilots may overload our CEs
  • Feel free to block the certificaiton DN if it is too annoying.