RAL Tier1 Experiments Liaison Meeting

Wednesday 11 Oct 2023, 13:30 → 15:00 Europe/London

Access Grid (RAL R89)

14:00 → 14:01 Major Incidents Changes

14:01 → 14:02 Summary of Operational Status and Issues

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore (Science and Technology Facilities Council STFC (GB)), Kieran Howlett (STFC RAL)

Weekly Report 11 October 2023.docx

Weekly Report 11 October 2023.pdf

14:02 → 14:03 GGUS /RT Tickets

https://tinyurl.com/T1-GGUS-Open
https://tinyurl.com/T1-GGUS-Closed

14:04 → 14:05 Site Availability

https://lcgwww.gridpp.rl.ac.uk/utils/availchart/

https://cms-site-readiness.web.cern.ch/cms-site-readiness/SiteReadiness/HTML/SiteReadinessReport.html#T1_UK_RAL

http://hammercloud.cern.ch/hc/app/atlas/siteoverview/?site=RAL-LCG2&startTime=2020-01-29&endTime=2020-02-06&templateType=isGolden

14:05 → 14:06 Experiment Operational Issues

14:15 → 14:16 VO Liaison CMS

Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))

(Writing on Tuesday morning, 10th Oct)

Failing SAM tests last week on Monday - Thursday inclusive, as well as Saturday. On Wed and Thur, this was due to the squids having been taken down (relating to the CVMFS-not-updating issue). The rest of the time it was the usual insufficient gateways problem. I prevented CMS from going into drain during the week, as jobs were performing well. On Sunday morning we went into drain again, but I left it as it was, since the job pressure in the global system was low anyway.

Concerning the CVMFS-not-updating, this affected CMS Tier-0 Processing jobs. These are new types of jobs, that previously only ran at Tier-0, but now also run at Tier 1s. I believe they do prompt processing. They needed the very latest version of CMSSW so were unable to run at RAL until CVMFS was fixed.

At time of writing, CMS job pressure is improving, but we are not back to the usual number of running cores worldwide.

 

14:16 → 14:17 VO-Liaison ATLAS

Speakers: James William Walder (Science and Technology Facilities Council STFC (GB)), Jyoti Prakash Biswal (Rutherford Appleton Laboratory)

ECHO_Today.pdf

• GGUS-163297:
  • RAL-LCG2: deletion errors; in progress.
  •  RAL-LCG2-ECHO_DATADISK is back to be below its threshold (the dashed lines; the plot is also attached). Let's wait and watch for a few more days.
  • Ticket was updated yesterday. 

 

• Grid is draining again today - presumably due to lack of tasks.
  • RAL saw a reduction in jobs around 06:00 hrs.

 

• The RAL CVMFS issue and the fix have been discussed in most of the meetings this week. 

 

• RAL-LCG2 and RAL-LCG2_ECHO continue with scheduled downtime (04 Oct-01 Nov).
  • Host intentionally disconnected during network reconfiguration. 

14:20 → 14:21 VO Liaison LHCb

Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)

alice_tape_se_usage.png

1. cvmfs issues caused a lot of job failures last week.
2. Number of running jobs was low last week. Should be more or less OK this week. This is not RAL's problem, this is lack of production requests from LHCb.

14:25 → 14:28 VO Liaison LSST

Speaker: Timothy John Noble (Science and Technology Facilities Council STFC (GB))

14:30 → 14:31 VO Liaison Others

14:31 → 14:32 AOB

14:32 → 14:33 Any other Business

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore (Science and Technology Facilities Council STFC (GB))