Speaker
Description
Small neural networks are an important class of FPGA workload. Many scientific and edge applications run small models under tight latency budgets, including high-energy-physics triggering, network-intrusion detection, and keyword spotting. When the model is small enough to keep on-chip, FPGAs can deliver low-latency, real-time inference. Two widely-used toolchains approach deployment differently. FINN compiles each network into a custom spatial dataflow accelerator. Vitis AI runs networks on the DPU, a fixed general-purpose overlay. The two trade off differently on throughput, resource use, precision flexibility, and integration effort. Yet no existing comparison can directly inform that choice: the few prior studies compare at mismatched precision and operating points, so the framework’s own effect is never isolated, leaving a practitioner with no controlled basis for deciding which to use.
We compare FINN and Vitis AI on a resource-constrained Zynq UltraScale+ device (ZU3EG), holding precision (INT8) and operating point (batch-1) fixed. Our models are a family of small, shallow CNN and MLP image classifiers, compact networks suited to embedded inference. We use MNIST as a standard workload at this scale, and a residual network (ResNet-8) on CIFAR-10 as a fit probe, so that the framework is the only variable. We report performance and resource utilization, and analyze iso-area and iso-performance tradeoffs.
Neither framework dominates. Vitis AI’s overlay delivers nearly flat throughput and model-independent resource use. FINN’s dataflow approach is sensitive to network structure. Its resource cost tracks the peak per-layer compute more than the parameter count. The bounding resource (logic versus on-chip memory) shifts with folding even within a single network. The overlay runs at a higher clock, yet through spatial parallelism FINN reaches higher throughput on a small CNN folded aggressively, so frequency does not drive the comparison. Latency behaves the same way: the overlay's per-inference latency is nearly flat across model sizes. The dataflow approach is more variable, depending on how aggressively the model is folded. On a residual network at INT8, a sweep of folding configurations finds no point that fits the constrained device, busting on logic at high throughput targets and on-chip memory at low ones, while the overlay runs it directly.
These results give practitioners deploying small networks a basis for choosing between dataflow and overlay deployment, and clarify which differences are intrinsic to the two paradigms versus specific to a toolchain, device, or model.
| Do you plan to submit a 4-page extended abstract on OpenReview (only for Presentations/Posters)? | Yes |
|---|