RAL Tier1 Experiments Liaison Meeting

Wednesday 28 Feb 2024, 13:30 → 15:00 Europe/London

Access Grid (RAL R89)

14:00 → 14:01 Major Incidents Changes

14:01 → 14:02 Summary of Operational Status and Issues

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore, Kieran Howlett (STFC RAL)

Weekly Report 28 February 2024.docx

Weekly Report 28 February 2024.pdf

14:02 → 14:03 GGUS /RT Tickets

https://tinyurl.com/T1-GGUS-Open
https://tinyurl.com/T1-GGUS-Closed

14:04 → 14:05 Site Availability

https://lcgwww.gridpp.rl.ac.uk/utils/availchart/

https://cms-site-readiness.web.cern.ch/cms-site-readiness/SiteReadiness/HTML/SiteReadinessReport.html#T1_UK_RAL

http://hammercloud.cern.ch/hc/app/atlas/siteoverview/?site=RAL-LCG2&startTime=2020-01-29&endTime=2020-02-06&templateType=isGolden

14:05 → 14:06 Experiment Operational Issues

14:15 → 14:16 VO Liaison CMS

Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))

On the whole DC24 went well. RAL did particularly well as a source site; as a destination we were not so great, but improvements on the gateways (tunings, ECN value) were made in the last few days. See attached slides. Deletion rate was an issue. 

SAM tests: Token tests were intermittently red before DC24 and even worse during DC24 busy periods (failing all the time). However, after DC24 the tokens were green until yesterday afternoon. Jyothish said he moved the redirector away from round-robin at around that time the failures restarted.

Ticket just received on overloading CPU at RAL (CMS are trying to roll this out over all T1s). 

14:16 → 14:17 VO-Liaison ATLAS

Speakers: James William Walder (Science and Technology Facilities Council STFC (GB)), Jyoti Prakash Biswal (Rutherford Appleton Laboratory)

• GGUS-165358 ("Deletion attempts at RAL-LCG2 take a long time") was solved/closed on 19 February.
  • RAL DATADISK full pledge increase (2024/2025) -- ATLDDMOPS-5681.

 

• DC24 is over (last Friday)!
  • FTS was the major bottleneck. 
  • The LHCOPN issue led to lower transfer rates at RAL. 
  • On the last day, RAL performed quite well (this is after LHCOPN was fixed)!
  • Post-DC24 CERN-T1 tests are foreseen (the dates are not there yet).

 

• Data Carousel and RAL:
  • A new production using heavily data carousel has started => RAL has been put back, resulting in recent traffic.
    
  • It seems RAL has much larger staging activity than any other T1! Maybe, due to its profile. 
    

 

• Areas to be covered in the ATLAS talk @GridPP.
  

 

• Pending: eLog access.

14:20 → 14:21 VO Liaison LHCb

Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)

alice_dc24.png

LHCB:

• DC24 is over
  • Target rate for writing part is not achieved
  • Target rate for reading part is achieved
  • 4 Files were lost, most probably due to FTS issue
• Problems with WGProd jobs due to xrootd-proxy memory excess on 2021 Gen on Monday
  • Turning prefetch off might help

 

Alice:

• DC24 is over
  • Target rate was 100MB/s, 530MB/s achieved
• 2023 Heavy Ion data export is over
  • Its export was part of the DC24
  • Since data is exported directly to Antares, is its network going to be upgraded (i.e. moved to LHCOPN/LHCONE) at some point?
• TPC issue
  • Trying to mitigate it by setting up TCP keepalive feature. Unsuccessful so far.

14:25 → 14:28 VO Liaison LSST

Speaker: Timothy John Noble (Science and Technology Facilities Council STFC (GB))

14:30 → 14:31 VO Liaison Others

14:31 → 14:32 AOB

14:32 → 14:33 Any other Business

Speakers: Brian Davies (Lancaster University (GB)), Darren Moore