1. We upgraded Kubernetes two major versions from 1.21 to 1.23
  2. We upgraded HTCondor to 9.0.13 with OSG 3.6 on head/login nodes
  3. We did a yum update of all packages on the AF login and head nodes, including the latest mainline Kernel from ELRepo
  4. We are still upgrading workers in the background 
  5. We deferred the CephFS upgrade from v16 (Pacific) to v17 (Quincy) - we found 1 node (c001) with what seems to be a hardware error - all disks are reporting "I/O error" trying to mount them. Need to get the cluster clean before we upgrade major version.