Conference on Computing in High Energy and Nuclear Physics

Physics Data Forge: Unveiling the Power of I/O Systems in CERN’s Test Infrastructure

24 Oct 2024, 16:33

18m

Room 1.B (Medium Hall B)

Talk Track 1 - Data and Metadata Organization, Management and Access Parallel (Track 1)

Speaker

Guilherme Amadio (CERN)

Description

Remote file access is critical in High Energy Physics (HEP) and is currently facilitated by XRootD and HTTP(S) protocols. With a tenfold increase in data volume expected for Run-4, higher throughput is critical. We compare some client-server implementations on 100GE LANs connected to high-throughput storage devices. A joint project between IT and EP departments aims to evaluate RNTuple as a replacement for ROOT’s TTree format, with a focus on verifying the scalability of EOS storage using a new data format. Based on this project we run a large-scale experiment to stress CERN’s CPU, network and memory by using massively parallel analysis workflows and replicated datasets in EOS physics instances.

Presentation materials

xrootd-chep24.pdf