With the emerging of more and more sophisticated machine learning models in high energy physics, optimising the parameters of the models (hyperparameters) is becoming more and more crucial in order to get the best performance for physics analysis. This requires a lot of computing resources. So far, many of the training results are worked out in a personal computer or a local institution cluster, which prevents people from going to a wider search space or testing more brave ideas. We minimise the obstacle by implementing a hyperparameter optimisation (HPO) infrastructure into the ATLAS Computing Grid. Users submit one task of HPO and the Grid will take care of the optimisation procedure to return the best hyperparameter back to users. I will discuss about using High Performance Computers under this context as well.