NGT - Openlab "Optimising Floating Point Precision" Workshop

Contribution List

25. Welcome Session

Stefan Roiser (CERN)

01/07/2025, 14:00

24. Problem statement from experiments and SFT

Vincenzo Innocente (CERN)

01/07/2025, 14:10

2. Problem statement from Theory Department

Jacob Finkenrath

01/07/2025, 14:40

3. Discussion - Questions

01/07/2025, 14:50

4. Floating Point Emulation in NVDIA Math Libraries

Samuel Rodriguez (NVidia)

01/07/2025, 15:30

The trends in computer architecture, primarily driven by AI-based applications (most recently, large language models), has led to a rapid increase in the reduced- and mixed-precision computing capabilities of GPUs. These processors demonstrate an outsized power-efficiency (FLOPS/watt) advantage over systems almost exclusively focused upon native single- and double-precision arithmetic. Thus,...

5. Advancing AI with AMD: Open Source, Sovereign Innovation, and the Latest in CPU & GPU Performance

Joerg Roskowetz, Sayed Maudodi

01/07/2025, 16:15

12. VXP: Extended Precision Accelerator

Eric Guthmuller (CEA France)

01/07/2025, 16:45

in this talk, we propose a RISC-V-based accelerator aimed at extended precision computing for scientific computing applications. Furthermore, we show how it can help improving convergence of iterative solvers in real use cases. Lastly, we present details about our hardware implementations and results obtained on real silicon prototypes.

6. Extended Precision in Convex Optimisation

David Herrera-Marti (CEA France)

01/07/2025, 17:05

Semidefinite Programming is a matrix-form generalisation of linear programming, and is typically tackled using Interior Point Methods. These methods are of iterative nature and at each step, a matrix inversion needs to be performed. For small or sparse matrices, direct methods like sparse Cholesky factorisation are used. For dense matrices of larger size, like the ones that arise in convex...

7. Discussion - Questions

01/07/2025, 17:25

8. Using physics knowledge to improve numerical stability

Enrico Bothmann (CERN)

02/07/2025, 09:00

The numerically stable evaluation of scattering matrix elements near the infrared limit of gauge theories is of great importance for the success of collider physics experiments. We present a novel algorithm that utilizes double precision arithmetic and reaches higher precision than a naive quadruple precision implementation at smaller computational cost. The method is based on physics-driven...

9. Double-double for virtual amplitude evaluation

Vitaly Magerya (CERN)

02/07/2025, 09:30

Two-loop virtual amplitudes are one of the key ingredients of an NNLO cross-section calculation. In this talk I would like to describe the precision requirements of evaluating such amplitudes via sector decomposition and quasi-Monte Carlo integration, and to report on satisfying them using double-double floating point number implementation within pySecDec on CPU and GPU.

Based on:...

10. An overview of mixed precision strategies for scientific computing

Theo Mary (Computer Lab of Paris 6 (Lip6))

02/07/2025, 10:00

The increasing support of lower precision arithmetics in hardware provides new opportunities for high performance scientific computing. However, even though low precision arithmetics can provide significant speed, communication, and energy benefits, their use in scientific computing poses the challenge of preserving the accuracy and stability of the computation. To address this issue, a...

26. Adaptive Floating-Point Quantization for Efficient Neural Networks

Nicolo Ghielmetti (CERN)

02/07/2025, 10:30

The rapid growth of deep learning models, particularly Large Language Models (LLMs), which have increased their parameter counts nearly tenfold annually since 2018, has intensified the need for more efficient, power-aware deployment strategies. Quantization is a widely adopted technique for reducing the computational and memory footprint of neural networks by lowering numerical...

17. Discussion - Questions

02/07/2025, 10:50

18. Mixed precision ab initio tensor network state methods adapted for NVIDIA. Blackwell technology via emulated FP64 arithmetic

Ors Legeza (Wigner Research Centre for Physics, Hungary)

02/07/2025, 11:35

An overview of recent advances in tensor network state (TNS) methods are
presented that have the potential to broaden their scope of application
radically for strongly correlated quantum many body systems. Novel
mathematical models for hybrid multiNode-multiGPU parallelization on
high-performance computing (HPC) infrastructures will be discussed.
Scaling analysis on NVIDIA DGX-A100 and...

11. TNL: Numerical Library for Modern Parallel Architectures

Thomas Oberhuber (Czech Technical University in Prague)

02/07/2025, 12:05

TNL (www.tnl-project.org) is a collection of building blocks that facilitate the development of efficient numerical solvers and HPC algorithms. It is implemented in C++ using modern programming paradigms in order to provide a flexible and user-friendly interface similar to, for example, the C++ Standard Template Library. TNL provides native support for modern hardware architectures such as...

23. Float32 Expansions – A Possible Answer for Scientific Computing in the Era of AI-Driven GPU Development

František Stloukal (Czech Technical University in Prague)

02/07/2025, 12:20

In recent years, the emergence of large language models has led GPU vendors to prioritize performance improvements for lower-precision arithmetic, often at the expense of continued development for Float64. Meanwhile, scientific computing has increasingly relied on GPGPU acceleration, where double precision is still essential. Multi-word expansions for single-precision floating point numbers...

13. Discussion - Questions

02/07/2025, 12:35

14. Floating-Point Error Estimation Using Automatic Differentiation

Vassil Vasilev (Princeton University)

02/07/2025, 14:20

Floating-point errors highlight the inherent limitations of finite-precision computing, and if left unaddressed, they can lead to severe consequences. In high-precision applications, accurately quantifying these uncertainties is essential. Various approaches have been explored to tackle floating-point errors, including increasing numerical precision, employing compensation algorithms, and...

15. Precision auto-tuning and control of accuracy in high performance simulations

Fabienne Jézéquel (LIP6, Sorbonne Université)

02/07/2025, 14:50

In the context of high performance computing, new architectures, becoming more and more parallel, offer higher floating-point computing power. Thus, the size of the problems considered (and with it, the number of operations) increases, becoming a possible cause for increased uncertainty. As such, estimating the reliability of a result at a reasonable cost is of major importance for numerical...

19. Experiences with CADNA and the Madgraph5 Event Generator

Stephan Hageboeck (CERN)

02/07/2025, 15:20

This talk presents a summer student project that explored the numerical stability of MadGraph5 using CADNA. It focuses on how CADNA’s warning system and its ability to quantify floating-point precision were used to assess whether MadGraph5 can operate reliably with single-precision floating-point numbers.

16. Emulating Matrix Multiplication Using Mixed-Precision Computation

Katsuhisa Ozaki (Shibaura Institute of Technology)

02/07/2025, 16:20

This talk introduces a method for emulating matrix multiplication through mixed-precision computation. As exemplified by the Matrix Engine on GPUs, low-precision arithmetic can be performed significantly faster than conventional FP32 or FP64 operations. We present Ozaki Scheme I and II, which leverage low-precision arithmetic to achieve accuracy comparable to standard FP64, and discuss their...

20. Discussion - Questions

02/07/2025, 16:50

22. Closing Session

02/07/2025, 17:10

21. Closing Session

1. Problem statement from experiments and SFT