November 29, 2021 to December 3, 2021
Virtual and IBS Science Culture Center, Daejeon, South Korea
Asia/Seoul timezone

SOFIE: C++ Code Generation from ROOT/TMVA for Fast Deep Learning Inference

contribution ID 732
Not scheduled
Walnut (Gather.Town)



Poster Track 1: Computing Technology for Physics Research Posters: Walnut


Sitong An (CERN, Carnegie Mellon University (US))


Deep neural networks are rapidly gaining popularity in physics research. While python-based deep learning frameworks for training models in GPU environments develop and mature, a good solution that allows easy integration of inference of trained models into conventional C++ and CPU-based scientific computing workflow seems lacking.

We report the latest development in ROOT/TMVA that aims to address this problem. This new framework takes externally trained deep learning models in ONNX format or Keras and PyTorch native formats, and emits C++ code that can be easily included and invoked for fast inference of the model, with minimal dependency on linear algebra libraries only. We provide an overview of this current solution for conducting inference in C++ production environment and discuss the technical details with examples of the generated code.

More importantly, we present the latest and significant updates of this framework in supporting commonly used deep learning architectures, such as convolutional and recurrent networks, as well as a new capability to store deep learning models in ROOT format.

Furthermore, we demonstrate its current capabilities with benchmarks in evaluating popular deep learning models like resnet against popular deep learning inference tools, such as ONNXRuntime.


Previous CHEP 2021 proceeding that covers only basic NN and benchmark:


This is a significant and extensive update for SOFIE that vastly expands our supported NN models and functionalities. It is an important step that promises a practical solution for physicists looking to integrate their NN into their C++ workflow.

Speaker time zone Compatible with America

Primary authors

Sitong An (CERN, Carnegie Mellon University (US)) Lorenzo Moneta (CERN) Sanjiban Sengupta (IIIT Bhubaneswar) Ahmat Hamdan (ISSEA, Cameroon) Federico Sossai (University of Padova) Aaradhya Saxena (Indian Institute of Technology, Roorkee)

Presentation materials