Sep 12 – 16, 2022
OAC conference center, Kolymbari, Crete, Greece.
Europe/Athens timezone

Synthesization of Fast Gradients with Enzyme

Sep 15, 2022, 11:35 AM
20m
OAC conference center, Kolymbari, Crete, Greece.

OAC conference center, Kolymbari, Crete, Greece.

Talk Computer Science Progress in Computer Science

Speaker

William Moses (Massachusetts Institute of Technology)

Description

The synthesization of fast reverse- and forward-mode gradients is the key to many algorithms in scientific computing such as meta-learning, optimization, uncertainty quantification, stability analysis, machine learning-accelerated simulations, and end-to-end learning. Enzyme is an automatic differentiation tool which in difference to other tools is built into the LLVM compiler toolchain to be able to benefit from optimizations of the compiler synthesizing gradients for the optimized compiler representation of programs to unlock the significant performance gains this affords. Thus generating high performance gradients of any language going to the LLVM intermediate representation such as C/C++, Fortran, Julia, Rust, and others.
In this talk we will present how Enzyme is able to generate these gradients for forward- and reverse-mode for scientific programs which contain GPU-code (CUDA & ROCm) and parallelism (Julia tasks, MPI & OpenMP). In addition we will present how Enzyme is able to vectorize computation inside and outside of differentiation to further accelerate scientific workflows.

Primary authors

Ludger Paehler (Technical University of Munich) Tim Gymnich (Technical University of Munich) William Moses (Massachusetts Institute of Technology) Valentin Churavy (Massachusetts Institute of Technology)

Presentation materials