RAL Tier1 Experiments Liaison Meeting

Wednesday 24 Jun 2026, 13:30 → 15:50 Europe/London

Access Grid (RAL R89)

13:30 → 13:34 Site Operations

13:34 → 13:35 Experiment Operational Issues

13:35 → 13:40 ATLAS Operations Report

Speakers: Brij Kishor Jashal (Rutherford Appleton Laboratory), Jyoti Prakash Biswal (Rutherford Appleton Laboratory)

Brij ATLAS liason meeting notes

I'm at ATLAS week, unable to join
but I have added notes for today, please have a look at them

13:40 → 13:45 CMS Operations Report

Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))

Apologies again; Katy is at Networkshop in Nottingham giving a plenary this afternoon. 

There was a red spot in the SAM tests last Wed on the AAA and Jyothish thought this was related to new VMs for the service he created in proxmox interfering with the federation. 

Brian pointed out a few files repeatedly failing to go to Antares from Echo. Looking closely, this turned out to be just 4 files, with the failures starting in second week of May. The issue was that the checksum saved on Echo was incorrect. However the file itself was correct when downloaded. I let Jyothish know in case it was related to streamed checksums. I also created a ticket for CMS DM to invalidate the 4 replicas on Echo so that Rucio can re-transfer.

Jyothish now has access to the recycled Antares machines which will be the new AAA proxy machines. He is adding an XCache and we will see if we can improve performance for jobs outside RAL reading data from Echo. 

Reminder to self: Still to look at the /unmerged/ problem.

13:45 → 13:50 LHCb Operations Report

Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)

News:

• MR that fixes the problem with buggy xrootd 5.3.1 clients for LHCb software is finally merged and deployed!
  • 17 previously affected DaVinci versions are now linked to xrootd 5.5.0
    • That means that user jobs relying on this versions will no longer fail at RAL (or any other site using xrootd server with TLS)
  • 1 version using v5.1.1 is still affected
    • No compatible override in cvmfs

Operational issues:

• Problems with MC job failure rate is still present, though to lesser extend as far as i can see
  • CPUTime increase applied to MCReconstruction jobs as well, that should help
• Issues with DFC last week + problems with DIRAC Optimizers on Sunday
  • Resulted in spikes of failed jobs + drain

• Power cut last Thursday
  • Farm rebooted (killing all the jobs), but storage survived

13:50 → 13:55 ALICE Operations Report

Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)

• IPv6 connectivity problem solved (see GSTSM-742)
• Excess data removed
• Would be nice to install MLSensor on our Gateways (GSTSM-756)

13:55 → 14:00 LSST Operations Report

Speakers: Thomas Birkett, Timothy Noble (Science and Technology Facilities Council STFC (GB))

• LSST meeting last week
  • Edinburgh was lovely - rained the whole time- how I wish for rain now
• Made good connections for pushing work forwards (hence the jobs last week) - Had more LSST jobs on CE01 than CMS!
  • Move to submitting with HTCondor - also a move from 'single core' jobs to multicore jobs using DAGs 
  • Some interesting job mixes and resource requirements may mean LSST need some fairshare / scheduling config for Batch Farm
  • The power outage didnt effect LSST too badly for jobs
    
• LSST software stack at RAL died with power outages due to storage mounting issues and corruption
  • Attempted to move to proxmox in faster timescale - 
    • Now broken as some packages / modules are not being friendly
    •  
• Very little moved to RAL last week:
  

14:00 → 14:01 Tier-1 Projects

14:05 → 14:10 XRootD Development

Speakers: Alexander Rogovskiy (Rutherford Appleton Laboratory), Jyothish Thomas (STFC)

XRootD 6.1.0 is in testing
- gfal2 xrootd plugin still not available for xrootd 6

CMS AAA new managers are in SAM tests
svc20 has been added to the new managers as well

svc{41,42} are the xcache AAA machines, currently in testing

 

14:15 → 14:20 Utilizing GPUs

Speakers: Dr Brij Kishor Jashal (Rutherford Appleton Laboratory), Thomas Birkett

14:25 → 14:45 Hardware deployment

Echo SN
WN
GPU

Speaker: Jacob Ward

14:45 → 14:46 AOB

14:50 → 14:59 Summary of Operational Status and Issues

Speakers: Brian Davies (Science and Technology Facilities Council STFC (GB)), Darren Moore, Thomas Birkett

Weekly Report 24 June 2026 (1).docx

Weekly Report 24 June 2026 (1).pdf

15:00 → 15:05 Any other Business

Speakers: Brian Davies (Science and Technology Facilities Council STFC (GB)), Darren Moore