RAL Tier1 Experiments Liaison Meeting

Wednesday 22 Nov 2023, 13:30 → 15:00 Europe/London

Access Grid (RAL R89)

14:00 → 14:01 Major Incidents Changes

14:01 → 14:02 Summary of Operational Status and Issues

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore (Science and Technology Facilities Council STFC (GB)), Kieran Howlett (STFC RAL)

Weekly Report 22 November 2023.docx

Weekly Report 22 November 2023.pdf

14:02 → 14:03 GGUS /RT Tickets

https://tinyurl.com/T1-GGUS-Open
https://tinyurl.com/T1-GGUS-Closed

14:04 → 14:05 Site Availability

https://lcgwww.gridpp.rl.ac.uk/utils/availchart/

https://cms-site-readiness.web.cern.ch/cms-site-readiness/SiteReadiness/HTML/SiteReadinessReport.html#T1_UK_RAL

http://hammercloud.cern.ch/hc/app/atlas/siteoverview/?site=RAL-LCG2&startTime=2020-01-29&endTime=2020-02-06&templateType=isGolden

14:05 → 14:06 Experiment Operational Issues

14:15 → 14:16 VO Liaison CMS

Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))

14:16 → 14:17 VO-Liaison ATLAS

Speakers: James William Walder (Science and Technology Facilities Council STFC (GB)), Jyoti Prakash Biswal (Rutherford Appleton Laboratory)

FZK-RAL_transfer_13-20Nov2023.jpg

• The HammerCloud test failures continue to occur, more or less on a daily basis. 
  • Sometimes leading to HammerCloud exclusion for shorter time period.

 

• The job failure rate has come significantly down w.r.t. early last week numbers. 

 

• FZK-KIT/RAL transfer issue has been fixed (GGUS-164115), transfers are much better now [plot attached].

 

•  The wn-2021-xma tranche of workernodes got upgraded to xrootd 5.6.3 for the gateway and proxy on 20 November. 
  • Keeping an eye on things. 

14:20 → 14:21 VO Liaison LHCb

Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)

Operational issues:

• Failed downloads from ECHO to local WNs (GGUS 164148 )
  • Problem seems to be caused by xrootd bug, fixed in newer versions
  • Upgrade has been rolled-out to 2021 gen, more to follow
• Failed transfers between ECHO and Antares (GGUS 164131)
  • Failures happen due to Antares restarts
    • No clear reason why restarts are happening
    • Hopefully, upgrade to EOS5 will help
• Slow checksums (GGUS 159137 )
  • Enhanced xrootd version was deployed to 3 gateways more to follow
  • First test results does not look very promising though..
• Redirector setup (GGUS 160671)
  • Finished, ticket closed
• Network issue between RALPP and RAL (GGUS 163876 )
  • Still present
• There were a few spikes of failed uploads last week
  • This is a little bit concerning, since redirector should have eliminated the problem

 

14:25 → 14:28 VO Liaison LSST

Speaker: Timothy John Noble (Science and Technology Facilities Council STFC (GB))

14:30 → 14:31 VO Liaison Others

14:31 → 14:32 AOB

14:32 → 14:33 Any other Business

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore (Science and Technology Facilities Council STFC (GB))