Speaker
Description
We introduce Noise Injection Node Regularization (NINR), a method that injects structured noise into Deep Neural Networks (DNNs) during the training stage, resulting in an emergent regularizing effect. We present both theoretical and empirical evidence demonstrating substantial improvements in robustness against various test data perturbations for feed-forward DNNs trained under NINR. The novelty of our approach lies in the interplay between adaptive noise injection and initialization conditions, such that noise becomes the dominant driver of dynamics at the start of training. Since this method simply requires the addition of external nodes without altering the existing network structure or optimization algorithms, it can be easily incorporated into many standard problem specifications. We observe improved stability against a range of data perturbations, including domain shifts, with the most dramatic improvement occurring for unstructured noise, where our technique outperforms existing methods such as Dropout or L2 regularization in some cases. Additionally, we show that desirable generalization properties on clean data are generally maintained. This method is well-suited for many physical scenarios where robust predictions are critical to neural network performance. Currently, we are employing this method to improve networks’ ability to discriminate between prompt and non-prompt photons in highly noisy processes in the most recent simulations of the ATLAS detector.
