19–25 Oct 2024
Europe/Zurich timezone

CVMFS: Pushing performance on highly parallel, many-core clients

24 Oct 2024, 17:09
18m
Room 2.B (Conference Room)

Room 2.B (Conference Room)

Talk Track 4 - Distributed Computing Parallel (Track 4)

Speaker

Laura Promberger (CERN)

Description

The CernVM File System (CVMFS) is an efficient distributed, read-only file system that streams software and data on demand. Its main focus is to distribute experiment software and conditions data to the world-wide LHC computing infrastructure. In WLCG, more than 5 billion files are distributed via CVMFS and its read-only file system client is installed on more than 100,000 worker nodes. Recent hardware trends have increased the usage of CVMFS in highly parallel environments. Nodes with more than 64 physical cores running concurrent workloads are common. These highly parallel, many-core workloads have exposed specific bugs and limitations of CVMFS in such environments. This contribution reports on the developments that address these issues, and presents new performance benchmarks on machines with 256 (virtual) cores.

Primary authors

Jakob Blomer (CERN) Laura Promberger (CERN) Mr Matt Harvey (Jump Trading) Mr Reza Naghibi (Jump Trading) Valentin Volkl (CERN)

Presentation materials