News:
- Pull request allowing separation of local and remote endpoints merged.
- So, we are ready to start using local gateways for writes to ECHO! Any objections to this?
Operational issues:
- Sandbox store problems at CERN last week, especially during the weekend
- Resulted in many spikes of rescheduled jobs and reduction of the number of running jobs across the grid
- Fixed now
- power outage in the ECHO SN rack on Monday evening
- Echo became sad, resulting in a spike of failed WGProduction jobs
- Intermittent ETF test failures for ECHO
- One of the gateways (
ceph-svc17) had *.s3.echo.stfc.ac.uk in its PTR record for the IPv4 address, as a result some requests were redirected to this incorrect endpoint
- Failed downloads from RAL to HLTFarm
- No external connectivity on HLTFarm WNs, therefore successful downloads are impossible. It's a bug in DIRAC -- these downloads should not have been attempted in the first place.