Speaker
Description
Machine learning (ML) models are increasingly being used in high-energy physics. However, the selection and training of these models frequently involves human intervention, extensive hyperparameter tuning, and consideration of data changes. These challenges become particularly pronounced when developing models for automated pipelines or fault-tolerant systems. We introduce a novel, automated approach for assessing and selecting robust ML models, specifically deep neural networks, where robustness is defined as the degree of performance loss variability for different training samples and initialisations.
Our method evaluates the variability in losses for multiple model instances and incorporates a meta-algorithm for selecting high-performing yet robust models.
We apply the model selection algorithm to neural networks with a few convolutional and fully connected layers (with a number of parameters no more than 30,000) for two regression problems. Overall, we systematically analysed 6,912 model architectures, training over 40,000 model instances, to find out how training sample size and initialisation of weights affect robustness. We demonstrate that selected models outperform Neural architecture search (NAS) utilising Bayesian optimisation in both robustness and performance, offering an effective strategy for reliable model deployment under challenging environments. The proposed approach is model-agnostic and suitable for integration into AutoML pipelines — an important step toward automated, scalable, and trustworthy ML in HEP and beyond.
| Would you like to be considered for an oral presentation? | Yes |
|---|
