Speaker
Description
Rendering algorithms simulate light by evaluating high-dimensional integrals that convert 3D scene descriptions into realistic images. Differentiable rendering turns this around: given one or more input observations (photographs or other measurements), it searches for a physical scene model that explains how these images were formed. This view casts many imaging tasks as inverse problems and has important applications in vision, graphics, and the natural sciences.
Differentiable light transport raises several unique challenges. Visibility introduces countless step discontinuities in the underlying integrals, which lead to incorrect gradients. Traditional tape-based automatic differentiation quickly exhausts memory because of the large amount of intermediate state. Efficient simulation further requires compilation to GPU “megakernels” that drive hardware ray tracing units. Finally, to be useful for domain experts, these simulations must remain highly customizable.
In this talk, I will present my group’s work on addressing these challenges, including Dr.Jit, a just-in-time compiler for differentiable Monte Carlo simulations, the Mitsuba renderer, and the nanobind Python binding library.