CVMFS Sync Meeting

Friday 6 Mar 2026, 11:00 → 12:30 Europe/Zurich

31/S-023 (CERN)

11:00 → 11:10 Operational status & service overview

• Current CVMFS service health (repositories, stratum-0/1, monitoring)
• Any incidents since last meeting
• Short-term operational risks or alerts

Got contacted by Luis about PDC. Ideas?

Fabrizio, we have made available some capacity in PDC fort LCG virtual machines (follow-up from an incident in MDC that impacted the only LCG area there).

I see IT CVMFS - LCG is one of the projects with access to those resources.

If you're interested, you can give it a try to spawn some LCG instances in PDC to add some resiliency in case any issue

If you want to start uisng it, you should request quota for the PDC region in that project, we can follow it up from there.

 

 

Conclusion:

S1 will have:
2 ironic in MDC
1 VM in PDC
1 VM in MDC
(ALL in the LCG network)

 

S0 will have:
2 VMs in MDC
2 VMs in PDC
(ALL in the LCG network)

S1-hpc will have
1VM in PDC
1VM in MDC

(ALL in the LCG network)

11:10 → 11:20 Issues & interventions follow-up

• Review of open issues
• Status of ongoing or planned interventions
• Decisions needed / actions agreed

Nobody answered to RQF3626318 (23/03). They want us to cache a new repo in the S1 and make it available on lxplus and CI. Proceed?

INC4875940 was left in alarm (well, there were no consequences yet, still...)

INC4870831 ams.cern.ch again stuck due to disk full. Seems they recognized it was their fault, twice in a week. Reported on Friday evening, solved by Enrico on Monday (thanks Enrico!)

INC4871912 network hiccup, would have been nice to close it after checking that the cvmfs log went right by itself

 

INC4884548 i can't understand why only this machine gives this issue, and not its sister... to me this makes little sense, as it is using swap with plenty of available ram
MiB Mem :  63772.9 total,    665.7 free,  37414.9 used,  26348.4 buff/cache
MiB Swap:   8192.0 total,     36.7 free,   8155.3 used.  26358.0 avail Mem

... could it be this?

To check the swappiness value

cat /proc/sys/vm/swappiness

To change the swappiness value A temporary change (lost on reboot) with a swappiness value of 10 can be made with

sudo sysctl vm.swappiness=10

11:20 → 11:30 Onboarding & knowledge transfer

• Onboarding progress for Hugo and Maria

• Open questions or unclear areas

• Documentation gaps or runbooks to improve

• Identify topics needing deeper walkthroughs (future sessions)

11:30 → 11:40 Collaborator input – SFT / Valentin

• Updates from SFT side
• Harmonization of the publishers infrastructure - progress to make it possible

11:40 → 11:45 Planning & priorities

• Short-term priorities until next sync

• Upcoming milestones or expected changes

• Topics to follow up

11:45 → 11:50 AOB