RAL Tier1 Experiments Liaison Meeting

Wednesday 12 Nov 2025, 13:30 → 15:50 Europe/London

Access Grid (RAL R89)

13:30 → 13:34 Site Operations

13:34 → 13:35 Experiment Operational Issues

13:35 → 13:40 ATLAS Operations Report

Speakers: Brij Kishor Jashal (Rutherford Appleton Laboratory), Jyoti Prakash Biswal (Rutherford Appleton Laboratory)

ATLAS liason meeting notes Brij

13:40 → 13:45 CMS Operations Report

Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))

FirstReadsAtRAL.pdf

FirstReadsAtRAL.pptx

Noticed CE SAM tests timing out again during the final test - for all UK sites (ARC or HTCONDOR) over the weekend. Suspected network or AAA redirectors in the UK. 

In the meantime AAA SAM tests started failing all the time. Did some restarts including all redirectors, which were successful briefly. But clearly a problem here. Following up on the  AAA line of enquiry with Jyothish. He suspects OOM killer. It has been a bit 'busy' meaning a few hundred MB/s, which has caused issues in the past, but Jyothish says this is new, possibly related to this https://github.com/systemd/systemd/issues/36529

Job performance remains good. Job failures mostly user analysis. 

13:45 → 13:50 LHCb Operations Report

Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)

Operational Issues:

• Problems with Sandbox SE at CERN last Thursday
  • Resulted in drain and a spike of rescheduled jobs 
  • All sites were affected
• Problems with transfers from Beijing to RAL yesterday GGUS:1001097
  • Beijing CRLs are SHA-1 signed, and this digest is disabled in EL9 by default
  • Fixed, ticket closed
• Noisy cvmfs ETF test
  • There is currently a bug in the test which results in occasional false warnings
  • WIP to fix it

13:50 → 13:55 ALICE Operations Report

Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)

Problems with external IPv6 connectivity for alice ECHO redirectors. Ticket: IT-25-43996.

13:55 → 14:00 LSST Operations Report

Speakers: Mathew Sims, Timothy Noble (Science and Technology Facilities Council STFC (GB))

• Dataset DC2 being retired
  • Replaced with DP2_prep - working on script to register the ancilaries
  • Raw data being sent when its decided what it being used by each site for the multisite approach
• RAL to be used as a source for LSST DP1 data to the Data Accesss Centres (analysis facilities for LSST)
• Fixed an issue last week of a file not in place that was preventing RC2 jobs from running - now running well

 

• Potential issues:
  
  • Metadata database running postgresql 16, new pipelies will require at least version 17 for some of the built in functions (asked which are needed)
    • DB team have a plan to update to 18 by April 2026 - this may be too slow to support LSST - finding out timeline from LSST

 

14:00 → 14:01 Tier-1 Projects

14:02 → 14:07 Anatares Upgrade

Speakers: George Patargias, Thomas Byrne

14:08 → 14:13 XRootD Development

Speakers: Alexander Rogovskiy (Rutherford Appleton Laboratory), Jyothish Thomas (STFC)

14:14 → 14:19 Utilizing GPUs

Speakers: Jyoti Prakash Biswal (Rutherford Appleton Laboratory), Thomas Birkett

Tier-1 Project Summary Presentation.pdf

Tier-1 Project Summary Presentation.pptx

14:20 → 14:25 SSD Storage Evaluation

14:25 → 14:26 AOB

14:27 → 14:36 Summary of Operational Status and Issues

Speakers: Brian Davies (Science and Technology Facilities Council STFC (GB)), Darren Moore

Weekly Report 12 November 2025.docx

Weekly Report 12 November 2025.pdf

14:45 → 14:50 Any other Business

Speakers: Brian Davies (Science and Technology Facilities Council STFC (GB)), Darren Moore