RAL Tier1 Experiments Liaison Meeting

Wednesday 5 Apr 2023, 12:30 → 14:30 Europe/London

Access Grid (RAL R89)

13:00 → 13:01 Major Incidents Changes

13:01 → 13:02 Summary of Operational Status and Issues

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore (Science and Technology Facilities Council STFC (GB)), Kieran Howlett (STFC RAL)

Weekly Report 05 April 2023.docx

Weekly Report 05 April 2023.pdf

13:02 → 13:03 GGUS /RT Tickets

https://tinyurl.com/T1-GGUS-Open
https://tinyurl.com/T1-GGUS-Closed

13:04 → 13:05 Site Availability

https://lcgwww.gridpp.rl.ac.uk/utils/availchart/

https://cms-site-readiness.web.cern.ch/cms-site-readiness/SiteReadiness/HTML/SiteReadinessReport.html#T1_UK_RAL

http://hammercloud.cern.ch/hc/app/atlas/siteoverview/?site=RAL-LCG2&startTime=2020-01-29&endTime=2020-02-06&templateType=isGolden

13:05 → 13:06 Experiment Operational Issues

13:15 → 13:16 VO Liaison CMS

Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))

Updated pledges (only partly done in an automated way):

Update on issues from last report (22nd March):

gsiftp write test failures - fixed by James W's observation of checksumming issues on gw7(?). 

Open access test - showed that CMS (and other VO) files are available to be recalled from Antares buffer by anyone, even without a cert proxy. Being addressed: https://stfc.atlassian.net/jira/servicedesk/projects/STAH/queues/custom/263/STAH-19

SAM tests requiring tokens - the test suite (webdav) is being run on ceph-dev-gw4 where JW has fixed the config to make the tests pass. Ready for discussion about rolling out to production.

WNs on LHCONE - by coincidence CMS did slot negotiation from FNAL unusually, rather than CERN, just after the change was made. FNAL was not configured to correctly assign jobs at RAL on LHCONE so although pilots were running, jobs fell to nearly zero. The switch back to negotiation at CERN fixed the problem, and I am told that the config at FNAL has been updated.

Shoveler: Spotted a problem with zero-value 'operation times' on many of the fstream entries. Likely due to a missing file open record. If you're interested see the ticket here: https://github.com/xrootd/xrootd/issues/1988

 

13:16 → 13:17 VO-Liaison ATLAS

Speakers: James William Walder (Science and Technology Facilities Council STFC (GB)), Jyoti Prakash Biswal (Rutherford Appleton Laboratory)

Made a request via GridPP Jira helpdesk to activate new pledges for Echo - no reply; Can we do it ? 

Compute; running at 2/3 pledge; seem to be effectively capped in the number of running multicore slots - can we progress this post Easter ? 

13:20 → 13:21 VO Liaison LHCb

Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)

Tickets:

• Vector read
  • Tests are still ongoing, no cms jost on the test WN so far
  • Code review is ongoing
• Remove environment variable
  • Seems like some LHCb jobs also need the variable, so we need a variable
    • Should we add cern's EOS host to the variable instead of removal?

Operational issues:

• Issues with FTS transfers PIC->RAL reappeared again last week
  • Confirmed routing issue at PIC
  • fixed
• Check for stub files has finished
  • ~69k stub/corrupted files were identified
  • 54 of them are registered in LHCb's file catalog, so they are lost
  • Some of the files (i.e. objects) are not present in weekly dumps, so can not be identified as dark data (they are "really dark").

13:25 → 13:28 VO Liaison LSST

Speaker: Timothy John Noble (Science and Technology Facilities Council STFC (GB))

13:30 → 13:31 VO Liaison Others

13:31 → 13:32 AOB

13:32 → 13:33 Any other Business

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore (Science and Technology Facilities Council STFC (GB))