Speaker
Description
The acceleration of machine learning and domain algorithm inference is increasing in importance as the LHC and other domains seek to improve reconstruction and analysis performance in extreme environments. At the same time, the geographically distributed computing infrastructure model is increasing in complexity, with the introduction of heterogeneous resources (HPC, HTC, cloud). There is corresponding tension between the demand for improved performance and the need for infrastructure flexibility. We will show the results of an initiative that aims to address such a dilemma by integrating two technologies in a real-world deployment. On one side, SONIC (services for optimized network inference on coprocessors) implements an efficient and cloud-friendly framework for GPU-accelerated inference as a service in scientific workflows. On the other side, interLink is a cloud-native solution to allow a Kubernetes cluster to seamlessly orchestrate workloads across supercomputers, HTC grid jobs, and cloud-hosted GPU VMs, through a minimal set of lightweight components. The result is the SONIC setup at Purdue University, where the orchestration of inference workloads is managed by a cloud-native stack deployed on a Kubernetes cluster, while the GPU-enabled inference servers are hosted at a different, Slurm-based cluster and made accessible to Kubernetes via interLink virtual nodes. We will highlight our experience operating such a system, the inference performance for a CMS experiment workflow, and the benefits of the technology stack, along with the current roadmap to address the main points of improvement.