Releases, Tickets, Testing, Board, ...
Cluster upgrades, capacity changes, rebalancing, ...
News from OpenStack block storage.
Filer Migration, CephFS/Manila status and plans.
Incident INC1973961: itnfs30 was not allowing OpenShift to mount anything. Loadavg was normal and zfs looked fine. Tons of processes of 'osadmin'. ~1000 zfs filesystems, which is abnormal. Alex Lossent found that a process on their side had created 800 volumes in a loop. He manually removed them, I restarted nfs-server, and things recovered.
I have, however, observed that the modification time of some data files remains unchanged when small amounts of data are appended to them. The modification time reported to the user stays the same even after days of running.
In the cases where this occured, the simulation program opened the data file with fopen() in "ab" mode, writes a few numbers and closes the file with fclose(). One can see that the file grows and it is possible to check that the data are correctly stored.
The same does not happen when larger amounts of data (a few KB at least) are appended. And it did not happen before the software update. There is no indication of data losses or data corruption.
Performance testing, HPC storage status and plans