RAL Tier1 Experiments Liaison Meeting

Wednesday 17 Jul 2024, 13:30 → 15:00 Europe/London

Access Grid (RAL R89)

14:00 → 14:01 Major Incidents Changes

14:05 → 14:06 Summary of Operational Status and Issues

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore, Kieran Howlett (STFC RAL)

Weekly Report 17 July 2024.docx

Weekly Report 17 July 2024.pdf

14:10 → 14:11 Experiment Operational Issues

14:15 → 14:16 VO Liaison CMS

Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))

The 'production' Shovler instance still hasn't been sending any monitoring information. Katy talked to the cloud team and they opened the 9993 port to the firewall but there are further changes to make. Katy also added the Shoveler config to the CMS WN - but we need the Shoveler to be working to see if it's sending data.

IPv6 connectivity of the AAA machines. A ticket has been sent to DI, which Katy cannot view but requested the service desk to progress it.

Job performance variable again - issues continuing to be investigated on the CMS side for particular campaigns with very low efficiency. Some failures, but not as bad as other T1s.

CMS job submission using EL9 queue only from what we (at RAL) can tell.

Antares - SAM tests using Xrootd (not webdav) failed Wed-Fri in common with several other VOs, e.g. NA62 see this ticket (now closed): https://ggus.eu/index.php?mode=ticket_info&ticket_id=167560 . The issue was related to broken VOMS extraction.

Still following up the handful of production transfers that have been failing for several weeks - CMS DM has a ticket and we have a list of files to invalidate in Rucio (which should then attempt a re-transfer via Echo).

Tom Birkett is testing on test01 the CE token setup. The production CEs now have token SAM tests (currently failing but not contributing to Site Status). 

Monday - a few CE SAM WN-mc tests failing due to stage-out; Tuesday - many more intermittent failures on CE tests for 'x509' (new test that arrived with the token test), applies to all CEs. Could both be related to the issues with WN-gateways reported by other VOs?

14:20 → 14:21 VO-Liaison ATLAS

Speakers: Brij Kishor Jashal (RAL, TIFR and IFIC), Jyoti Prakash Biswal (Rutherford Appleton Laboratory)

ATLAS Liaison notes

14:25 → 14:26 VO Liaison LHCb

Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)

6b7e77ad821d240ff8978b97bdb3f830.png

Tickets:

• Failed downloads due to proxy's issues
  
  • Looks like proxies are running out of memory
  • Can be reproduced on a dedicated WN with "tests" jobs and xrdcp downloads
  • To test: does turning prefetch off help?
    
    

Operational issues:

• xrootd bug follow-up?
• Is lcg-support@gridpp.rl.ac.uk e-mail still valid? It is used by LHCb ELOG to send notifications.
  
  • Do we want to receive this notifications?
    

14:30 → 14:33 VO Liaison LSST

Speaker: Timothy John Noble (Science and Technology Facilities Council STFC (GB))

14:35 → 14:36 VO Liaison APEL

Speaker: Thomas Dack

14:39 → 14:40 VO Liaison Others

Speakers: Alexander Rogovskiy (Rutherford Appleton Laboratory), Brij Kishor Jashal (RAL, TIFR and IFIC), Jyoti Prakash Biswal (Rutherford Appleton Laboratory), Katy Ellis (Science and Technology Facilities Council STFC (GB))

Katy - Ticket for NA62 issues with accessing Antares was resolved by George. Problem was with VOMS extraction. Also affected other VOs.

14:45 → 14:46 AOB

14:50 → 14:51 Any other Business

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore