Speaker
Description
We introduce a novel framework for optimization based on energy-conserving Hamiltonian dynamics in a strongly mixing (chaotic) regime. The prototype is a discretization of Born-Infeld dynamics, with a relativistic speed limit given by the objective (loss) function. This class of frictionless, energy-conserving optimizers proceeds unobstructed until stopping at the desired minimum, whose basin of attraction contains the parametrically dominant contribution to the phase space volume of the system. Building from mathematical and physical studies of chaotic systems such as dynamical billiards, we formulate a specific algorithm with good performance on machine learning and PDE-solving tasks. We analyze this and the effects of noise both theoretically and experimentally, performing preliminary comparisons with other optimization algorithms containing friction such as Adam and stochastic gradient descent.
