RAL Tier1 Experiments Liaison Meeting

Wednesday 13 Dec 2023, 13:30 → 15:00 Europe/London

Access Grid (RAL R89)

14:00 → 14:01 Major Incidents Changes

14:01 → 14:02 Summary of Operational Status and Issues

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore (Science and Technology Facilities Council STFC (GB)), Kieran Howlett (STFC RAL)

Weekly Report 13 December 2023.docx

Weekly Report 13 December 2023.pdf

14:02 → 14:03 GGUS /RT Tickets

https://tinyurl.com/T1-GGUS-Open
https://tinyurl.com/T1-GGUS-Closed

14:04 → 14:05 Site Availability

https://lcgwww.gridpp.rl.ac.uk/utils/availchart/

https://cms-site-readiness.web.cern.ch/cms-site-readiness/SiteReadiness/HTML/SiteReadinessReport.html#T1_UK_RAL

http://hammercloud.cern.ch/hc/app/atlas/siteoverview/?site=RAL-LCG2&startTime=2020-01-29&endTime=2020-02-06&templateType=isGolden

14:05 → 14:06 Experiment Operational Issues

14:15 → 14:16 VO Liaison CMS

Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))

A good week for SAM tests. Just the intermittent CVMFS test failures are still present, ticket is here: https://stfc.atlassian.net/browse/GS-260

Consistency test is still failing but the file is now available, I downloaded it, gunzipped and can read it fine. The error is still the same - further investigations needed.

CMS doing a pre-DC24 test today to stress Rucio and FTS. Sites should see some 'Data Challenge' traffic but rates should not be large. File size should be on the small side. Also keeping an eye on deletions.

Further tests needed for RAL re: DC24. Discussed with Alastair and Jyothish (separately). Have a plan to start with individual VO transfer tests from multiple sites to Echo, working up to multi-VO tests to look at the contention between CMS and ATLAS.

Tokens at RAL for CMS - worked with Jyothish again yesterday and made some progress. We can now produce a situation where SAM tests for tokens are passing on the dev gateway and a user (Katy) token can access CMS files but not dteam files. However, the user's token scope needs to contain the part of the prefix called 'restricted_path' in the token config on the gateway. Discussion with Stephan Lammel for CMS on the ticket suggests this is not ideal. 

 

14:16 → 14:17 VO-Liaison ATLAS

Speakers: James William Walder (Science and Technology Facilities Council STFC (GB)), Jyoti Prakash Biswal (Rutherford Appleton Laboratory)

• Network forward look form to be filled/checked (if not done so yet).

 

• Pre-DC24 retest @RAL dates are yet to be confirmed.
  • To be discussed over the "GridPP Data Challenge 24" Mattermost channel.
  • Likely to be sometime before the ATLAS S&C week (05-09 February, 2024), which is one week prior to DC24.

 

• HammerCloud exclusion on 11 December: 19:20-19:50 hrs.
  
  • lcg2360.
  • Error description: pilot, 1305: Error: tool unavailable for this platform: lcgenv /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/swConfig/root/postSetup-Linux.sh: line 13: /bin/root-config: No such file or directory trans, 200: Log file not transferred to destination.
  • Confirmed that no functional test failures on external gateways during that time .

 

• Storage Resource Reporting (SRR) checks performed.
  • Antares link is now updated on RAL-LCG2 CRIC page.
  • Echo: https://s3.echo.stfc.ac.uk/srr/storagesummary.json
  • Antares: https://s3.echo.stfc.ac.uk/srr/ctastoragesummary.json
  • Both are up to date.
  • Link to SRR documentation.

 

• RAL-LCG2_TEST is fine; usually flat with about 7.50k jobs. 

 

• SWAP issues:
  
  • This link behaves weirdly most of the time.
  • Difficult to view an issue from past. 
  • There were som issues last week, not higher in number. 

14:20 → 14:21 VO Liaison LHCb

Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)

Operationsl issues:

• Network issue between RALPP and RAL
  • Still present
• Slow checksums
  • Last week we discussed a way to make xrootd request checksum from file attributes, without running the scripts
    
    • Two things are required for this: fix logic for setting "local checksum" flag, and remove stale checksum check
    • The latter can be done in a dedicated checksum library

14:25 → 14:28 VO Liaison LSST

Speaker: Timothy John Noble (Science and Technology Facilities Council STFC (GB))

14:30 → 14:31 VO Liaison Others

14:31 → 14:32 AOB

14:32 → 14:33 Any other Business

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore (Science and Technology Facilities Council STFC (GB))