Fast Machine Learning for Science Conference 2026

pyrtlnet: An End-to-End Python Workflow for Quantized Inference Hardware Development

Not scheduled

12m

Presentation Contributed Talks

Speaker

Timothy Sherwood (UC Santa Barbara)

Description

As quantized neural network inference moves from cloud to edge, the demand for custom hardware accelerators tailored to specific models, precisions, and deployment constraints continues to grow. While frameworks for training and deploying quantized models in software are mature, the path from a trained model to working custom hardware remains steep and requires expertise spanning machine learning frameworks to hardware description languages, synthesis toolchains, and FPGA deployment flows. To ease innovation across these different layers of abstraction, we present pyrtlnet, an open-source Python-based framework that provides an end-to-end workflow from quantization-aware training in TensorFlow to custom inference hardware running on an FPGA, all without ever leaving Python.

The core idea behind pyrtlnet is to provide a sequence of matched workflows that progressively move from software to hardware: a TensorFlow quantization-aware training flow produces 8-bit integer weights and 32-bit integer biases; a LiteRT-based inference flow validates the quantized model in software; a NumPy reimplementation of that same inference exposes the quantization math in detail; a PyRTL hardware description reimplements the NumPy version as synthesizable digital logic; and finally a PYNQ-based deployment flow targets the design to an FPGA with AXI data transport. Because each workflow reimplements the same computation at a different level of abstraction, researchers can verify correctness at every step and build intuition for how software-level decisions (such as quantization bit-widths or layer structure) manifest in hardware cost and behavior. The generated hardware is easy to inspect, modify, and extend, so researchers can experiment with new layer types, quantization schemes, or dataflow organizations without navigating the complexity of industrial HDL toolchains. The result is a system that enables researchers to move from a trained model to synthesized hardware in an afternoon, opening the door to rapid co-exploration of model architecture and hardware design.