Fast Machine Learning for Science Workshop 2023

Neural signature kernels as infinite-width-depth limits of controlled ResNets

Not scheduled

5m

Blackett Laboratory, Lecture Theatre 1 (Imperial College London)

Lightning Talk Contributed Talks Contributed Talks

Speaker

Cristopher Salvi (Imperial College London)

Description

I will consider randomly initialized controlled ResNets and show that in the infinite-width-depth limit and under appropriate rescaling of weights and biases, these architectures converge weakly to Gaussian processes indexed on path-space and with kernels realised as solutions of certain data-dependent PDEs, varying according to the choice of activation function. In the special case where the activation function is the identity, the equation for the limiting kernel reduces to a variant of the wave equation (of Goursat type) that agrees with the one of the original signature kernel. Based on joint work with Nicola M. Cirone and Maud Lemercier.