15–18 Oct 2024
Purdue University
America/Indiana/Indianapolis timezone

SONIC: A Portable framework for as-a-service ML serving

15 Oct 2024, 14:45
15m
Steward Center 306 (Third floor) (Purdue University)

Steward Center 306 (Third floor)

Purdue University

128 Memorial Mall Dr, West Lafayette, IN 47907
Standard 15 min talk Contributed talks

Speaker

Dmitry Kondratyev (Purdue University (US))

Description

Computing demands for large scientific experiments, including experiments at the Large Hadron Collider and the future DUNE neutrino detector, will increase dramatically in the next decades. Heterogeneous computing provides a solution enabling increased computing demands that pass the limitations brought on by the end of Dennard scaling. However, to effectively exploit Heterogeneous compute, software needs to be adapted, and resources need to be balanced. We explore the novel approach of Services for Optimized Network Inference on Coprocessors (SONIC) and present a strategy for optimized integration of heterogeneous coprocessors, including GPUs, FPGAs, Graphcore IPUs and others. Focusing on ML algorithms, we demonstrate how SONIC can be designed to dynamically allocate heterogeneous resources in an fully optimized mode. With the rapid adoption of deep learning models for core algorithms at big scientific experiments, we present a path towards rapid integration of deep learning models, and strategy for future large scale compute at big experiments including the CMS and ATLAS detectors at the Large Hadron Collider. We show our proposed path clears the way for substantially improved data processing by optimally exploiting resources while simultaneously increasing the bandwidth and overall computational power of these future experiments.

Focus areas HEP

Primary authors

Dmitry Kondratyev (Purdue University (US)) Garyfallia Paspalaki (Purdue University (US)) Haoran Zhao (University of Washington (US)) Javier Mauricio Duarte (Univ. of California San Diego (US)) Kelci Ann Mohrman (University of Florida (US)) Kevin Pedro (Fermi National Accelerator Lab. (US)) Miaoyuan Liu (Purdue University (US)) Miles Cochran-Branson (University of Washington (US)) Nhan Tran (Fermi National Accelerator Lab. (US)) Noah Paladino (Massachusetts Inst. of Technology (US)) Philip Chang (University of Florida (US)) Philip Coleman Harris (Massachusetts Inst. of Technology (US)) Shih-Chieh Hsu (University of Washington Seattle (US)) Stefan Piperov (Purdue University (US)) Yao Yao (Purdue University (US)) Yongbin Feng (Texas Tech University (US)) Yuan-Tang Chou (University of Washington (US))

Presentation materials