Speaker
Mr
Keegan Stoner
(Northeastern University)
Description
We apply a recent correspondence between neural networks and quantum field theory to study RG fixed points of single-layer neural networks with exponential activation. Many architectures with a biased linear output layer exhibit a universal fixed point at large cutoff, and for some architectures, another fixed point at low cutoff. These fixed points are demonstrated at second-order in the leading non-Gaussian coefficients of the distribution, which are defined using the techniques of Wilsonian effective field theory.
