Speaker
Description
The synthesization of fast reverse- and forward-mode gradients is the key to many algorithms in scientific computing such as meta-learning, optimization, uncertainty quantification, stability analysis, machine learning-accelerated simulations, and end-to-end learning. Enzyme is an automatic differentiation tool which in difference to other tools is built into the LLVM compiler toolchain to be able to benefit from optimizations of the compiler synthesizing gradients for the optimized compiler representation of programs to unlock the significant performance gains this affords. Thus generating high performance gradients of any language going to the LLVM intermediate representation such as C/C++, Fortran, Julia, Rust, and others.
In this talk we will present how Enzyme is able to generate these gradients for forward- and reverse-mode for scientific programs which contain GPU-code (CUDA & ROCm) and parallelism (Julia tasks, MPI & OpenMP). In addition we will present how Enzyme is able to vectorize computation inside and outside of differentiation to further accelerate scientific workflows.
