Speaker
Description
The ATLAS experiment at the CERN Large Hadron Collider (LHC) records and processes large amounts of data from proton-proton collisions. With the upcoming High-Luminosity LHC (HL-LHC), the data volume is expected to increase by more than an order of magnitude, posing new challenges for storage, data throughput, and analysis scalability.
Currently, all major production output formats support RNTuple. Performance studies using ATLAS data have already demonstrated substantial improvements in space usage and I/O performance.
The main goal of this work is to explore the potential benefits of switching from the default LZMA compression to alternative compression algorithms such as ZSTD or LZ4 and to study their impact on both file size and I/O throughput for AOD data. In this study, we focus specifically on AOD reading during the processing into derived, smaller data formats. This process is I/O-intensive and most frequently executed in multiprocessing mode workflows in ATLAS production. Our priority is to determine whether using different compression algorithms for the AOD stored in the RNTuple layout can provide measurable improvements in input throughput, and how these throughput changes compare to the corresponding differences in file size.
This work contributes to the ongoing effort to prepare ATLAS computing for the data-intensive HL-LHC and will be critical for supporting large-scale data analysis.