Speaker
Description
Classical minimization methods, like the steepest descent or quasi-Newton techniques, have been proved to struggle in dealing with optimization problems with a high-dimensional search space or subject to complex nonlinear constraints, in addition to requiring continuous cost functions. For these reasons, in the last decade, the interest on metaheuristic nature-inspired algorithms has been growing steadily, due to their flexibility and effectiveness.
In this talk I will present a new python package which implements several metaheuristic algorithms (MHAs) for optimization problems. Unlike other available tools, it can be used not only for unconstrained problems, but also for problems subjected to inequality constraints and for integer or mixed-integer problems. Within the HEP community, a particularly interesting use case of MHAs is for the optimization of artificial neural networks (ANNs) and deep learning (DL) architectures, in terms of weights, hyper-parameters, and so on. This is indeed known to be one of the most challenging problems in machine learning and traditional gradient-based learning algorithms sometimes suffer from stucking at local minima and from the need of continuous cost functions. For these reasons, several authors have investigated the use of MHAs, even by combining some of them (hybrid MAHs), in optimizing the parameters involved in the training process of ANNs and, more recently, of DLs. In this talk I will also give an overview on this subject and show how to use the new package to integrate MHAs with ANN and DL architectures.
