CMS FTS transfers used gridftp following the CMS Rucio upgrade. Not just at RAL, but RAL was one site that saw this - thanks to Jyothish for noticing! The Rucio upgrade has now been rolled back and gridftp transfers have gone back to their usual rate (i.e. ~zero in FTS, and just a couple of transfers per bin in Vande monitoring, which are the WN uploads from RAL T1 batch farm).
Note: FTS have announced that they will end support on CERN CMS FTS for gridftp on May 7th!
Katy to move batch farm uploads at RAL from gridftp to davs ASAP.
Number of cores: other VOs are low on cores. CMS is not. In fact the 12k 'cap' appears to have been released if you look at CMS monitoring. However this does NOT appear in Vande, so some artefact, or possibly related to the pilot overloading.
It was pointed out today that the OPN is saturated and it is the IPv4 (WNs) that are drawing the data. There is corresponding rise in 'average data read' by CMS jobs. There is also a big jump in read time (possibly greater than proportional to data quantity), and a drop in efficiency taking us below the CMS T1 average.
Gateway migration: AAA gateway certificates were sorted out and all tests are green again as of this week. The issue was with the job that pulled the certificate was broken.
CMS jobs running on the test CEs / pre-prod farm => to be checked
Katy has pushed for EL9 jobs but CMS seem slow to act.