Jun 3 – 5, 2019
CERN
Europe/Zurich timezone

Using CVMFS on a distributed Kubernetes cluster - The PRP experience

Jun 3, 2019, 4:20 PM
20m
30/7-018 - Kjell Johnsen Auditorium (CERN)

30/7-018 - Kjell Johnsen Auditorium

CERN

190
Show room on map

Speaker

Igor Sfiligoi (UCSD)

Description

The Pacific Research Platform (PRP) is operating a Kubernetes cluster that manages over 2.5k CPU cores and 250 GPUs. The compute nodes are distributed over several locations, mostly in California, all connected with high speed networking of 10Gbps or higher.

In order to support OSG users, we were in need of providing CVMFS support in the Kubernetes cluster. Since user containers run unprivileged, CVMFS could not be used from the user containers themselves. The chosen path forward was to provide CVMFS as a mountable Kubernetes volume. We started with the CERN-provided CSI plugin, but had to make some changes to make it to work in our environment.

In this talk, we will detail the actual setup used, as well as the operational experience of the last 4 months. The perceived missing functionality will also be outlined.

Primary authors

Igor Sfiligoi (UCSD) Dima Mishin (UCSD) Dave Dykstra (Fermi National Accelerator Lab. (US))

Presentation materials