• GGUS-1002723 deletion failures on S3 storage
  • resolved. The VM of xrootd-s3gw had problems during the VMware cluster migration
• applied mitigation to both copy.fail and dirtyfrag exploits
• 5h downtime on 4/30 
  • Reconfigured dcache to use a unified base path for any/all accesses, via default variable dcache.root. 
  • applied firmware and software updates on all the nodes, and rebooted 
  • smooth ramping up after downtime ended. A short window of low transfer efficiency due to a zookeeper cluster issue, spotted and fixed within 30minutes. 
• 05/05: updated dcache from 11.2.3 to 11.2.4, smooth
• Remaining ~186 TB dark data on datadisk. 
      We think some/most of it is understood.
      Some of it must be leftover from the late 2024 dcache database problem
        some files are pinned on disk but have no pnfs entry.
      We will try to find those through database queries.