28 July 2020 to 6 August 2020
virtual conference
Europe/Prague timezone

Fast Entropy Coding for ALICE Run 3

30 Jul 2020, 09:20
20m
virtual conference

virtual conference

Talk 14. Computing and Data Handling Computing and Data Handling

Speaker

Michael Lettrich (Technische Universitat Munchen (DE))

Description

In LHC Run 3, the upgraded ALICE detector will record 50kHz Pb-Pb collisions using continuous readout. The resulting stream of raw data at ~3.5TB/s - a fiftyfold increase over Run 2 - must be processed with a set of lossy and lossless compression and data reduction techniques to decrease the data rate to storage to ~100GB/s without affecting the physics. This contribution focuses on lossless entropy coding for ALICE Run 3 data which is the final component in the compression stage. It describes the challenges imposed by the source data with a non-standard, patchy distribution of up to 25Bit per symbol. It then explains how we use rANS, a variant of Asymmetric Numeral System coders for compression with special emphasis on our adaptations and additions to the algorithm. Finally, we will show the first production ready rANS implementation for both CPU and GPU as part of the ALICE O$^{2}$ framework and demonstrate its capability of compressing data close to entropy at rates of up to 750 MB/s.

Primary author

Michael Lettrich (Technische Universitat Munchen (DE))

Presentation materials