29 November 2021 to 3 December 2021
Virtual and IBS Science Culture Center, Daejeon, South Korea
Asia/Seoul timezone

ROOT Files Improved with Extreme Compression

contribution ID 678
2 Dec 2021, 11:40
20m
S221-A (Virtual and IBS Science Culture Center)

S221-A

Virtual and IBS Science Culture Center

55 EXPO-ro Yuseong-gu Daejeon, South Korea email: library@ibs.re.kr +82 42 878 8299
Oral Track 1: Computing Technology for Physics Research Track 1: Computing Technology for Physics Research

Speakers

Gene Van Buren (Brookhaven National Laboratory) Jerome LAURET (Brookhaven National Laboratory) Ivan Amos Cali (Massachusetts Inst. of Technology (US))Dr Juan Gonzalez (Accelogic) Philippe Canal (Fermi National Accelerator Lab. (US))Mr Rafael Nunez Yueyang Ying (Massachusetts Inst. of Technology (US))

Description

For the last 7 years Accelogic pioneered and perfected a radically new theory of numerical computing codenamed "Compressive Computing", which has an extremely profound impact on real-world computer science [1]. At the core of this new theory is the discovery of one of its fundamental theorems which states that, under very general conditions, the vast majority (typically between 70% and 80%) of the bits used in modern large-scale numerical computations are absolutely irrelevant for the accuracy of the end result. This theory of Compressive Computing provides mechanisms able to identify (with high intelligence and surgical accuracy) the number of bits (i.e., the precision) that can be used to represent numbers without affecting the substance of the end results, as they are computed and vary in real time. The bottom line outcome would be to provide a state-of-the-art compression algorithm that surpasses those currently available in the ROOT framework, with the purpose of enabling substantial economic and operational gains (including speedup) for High Energy and Nuclear Physics data storage/analysis. In our initial studies, a factor of nearly x4 (3.9) compression was achieved with RHIC/STAR data where ROOT compression managed only x1.4.

As a collaboration of experimental scientists, private industry, and the ROOT Team, our aim is to capitalize on the substantial success delivered by the initial effort and produce a robust technology properly packaged as an open-source tool that could be used by virtually every experiment around the world as means for improving data management and accessibility.

In this contribution, we will present our efforts integrating our concepts of "functionally lossless compression" within the ROOT framework implementation, with the purpose of producing a basic solution readily integrated into HENP applications. We will also present our progress applying this compression through realistic examples of analysis from both the STAR and CMS experiments.

Significance

Completion of this work has the potential to enable the ability to save significant amount of storage space (or bandwidth) while not affecting noticeably the physics result quality.

Speaker time zone Compatible with America

Primary authors

Gene Van Buren (Brookhaven National Laboratory) Jerome LAURET (Brookhaven National Laboratory) Ivan Amos Cali (Massachusetts Inst. of Technology (US)) Dr Juan Gonzalez (Accelogic) Philippe Canal (Fermi National Accelerator Lab. (US)) Mr Rafael Nunez Yueyang Ying (Massachusetts Inst. of Technology (US))

Presentation materials