RAL Tier1 Experiments Liaison Meeting

Wednesday 15 Mar 2023, 12:30 → 14:30 Europe/London

Access Grid (RAL R89)

13:00 Major Incidents Changes

1 Summary of Operational Status and Issues

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore (Science and Technology Facilities Council STFC (GB)), Kieran Howlett (STFC RAL)

Weekly Report 15 March 2023.docx

Weekly Report 15 March 2023.pdf

2 GGUS /RT Tickets

https://tinyurl.com/T1-GGUS-Open
https://tinyurl.com/T1-GGUS-Closed

3 Site Availability

https://lcgwww.gridpp.rl.ac.uk/utils/availchart/

https://cms-site-readiness.web.cern.ch/cms-site-readiness/SiteReadiness/HTML/SiteReadinessReport.html#T1_UK_RAL

http://hammercloud.cern.ch/hc/app/atlas/siteoverview/?site=RAL-LCG2&startTime=2020-01-29&endTime=2020-02-06&templateType=isGolden

13:05 Experiment Operational Issues

4 VO Liaison CMS

Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))

During the Antares downtime, new CMS SAM tests started, including those for tape/xrootd endpoint. After the DT ended on Monday the Antares read test was failing because the files were not on buffer - Katy staged them, but have already seen them disappear from buffer. Need a way to keep them on buffer permanantly (3 files). Katy contacted Julien from the CTA team at CERN on Wednesday and hasn't had a reply at time of writing.

Another test checks 'open access' and it was found that someone without a CMS credential can stat and read CMS files from the buffer. CMS do not want this accessibiity - hence the test, which is currently failing. Looked into this with Tom Byrne and changed some permissions on the test files (as a test) but it's still failing - to be continued. 

However, it appears that so long as read/write tests are green, production data can read/write too. 

Katy to find out about the 'token' tests which are running and failing at all/many sites including RAL. 

In addition, the gridftp endpoint test for writest to Echo has been intermittently failing in the last week. I can't see why this is at the moment. 

Tape deletion campaign is planned. My estimate is that around 1.5PB will be removed from Antares. 

5 VO-Liaison ATLAS

Speakers: James William Walder (Science and Technology Facilities Council STFC (GB)), Jyoti Prakash Biswal (Rutherford Appleton Laboratory)

6 VO Liaison LHCb

Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)

• Open tickets
  • Connection timeouts to/from ECHO
    • two links affected: echo -> antares and PIC -> echo
    • First one due to restart script problems after DT
    • Second one due to PIC's (seemingly) routing issues
  • Vector Read
    • 870 user jobs successfully executed on two test WNs
    • Not a single vector read failure so far
    • Change Control page here
• Operations
  • Antares DT extension caused delay in removal of some buggy production files from antares
  • Problems with EOS after antares DT caused some transfer failures. They were erroneously attributed to ECHO problems in the ticket related to connection timeouts
  • New LHCbDirac release today (minor version update)
• Offtop
  • How to proceed with stub file search?

7 VO Liaison LSST

Speaker: Timothy John Noble (Science and Technology Facilities Council STFC (GB))

8 VO Liaison Others

13:31 AOB

9 Any other Business

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore (Science and Technology Facilities Council STFC (GB))