AAA was clearly sturggling with data accesses from remote sites. SAM tests were intermittently red and complaints were made by other sites. The files being accesed were 'premix library' files that are usually accessed only from CERN or FNAL. Katy found that these files had been stored on Antares, which is normal, but of course the only way they get to Antares is via Echo (multihop). By design of Rucio these files don't necessarily get deleted immediately...and then jobs around Europe start using them. This usage (popularity) may even reduce the chance of the files being removed, displaced by other files arriving on Echo. 

Jyothish increased the throttling level on the AAA gateways (previously was a high level of throttling on these machines allowing only 100MB/s in total over 3 gateways) and everything looks much better now. 

Job performance is commensurate with other T1s. However there is a drop in performance. Yet again CMS are running jobs that only use 1 core but request 8 - I complained again. 

CE tokens seem to be passing SAM tests quite well now, but I'm told there are still problems on the end of the test. 

Mini-DC for CMS UK sites next week - tests are for Tier 2s but Tier 1 will be used as a source and sink, making sure not to put unnecessary pressure on Tier 1 at this stage. (ATLAS already started their tests this week and will continue next week). 

TO DO: Get RAL-FTS to configure for CMS token access.