Streaming Architecture for Large-Scale Quantized Neural Networks on an FPGA-Based Dataflow Platform

Abstract

Deep neural networks (DNNs) are used by different applications that are executed on a range of computer architectures, from IoT devices to supercomputers. The footprint of these networks is huge as well as their computational and communication needs. In order to ease the pressure on resources, research indicates that in many cases a low precision representation (1-2 bit per parameter) of weights and other parameters can achieve similar accuracy while requiring less resources. Using quantized values enables the use of FPGAs to run NNs, since FPGAs are well fitted to these primitives; e.g., FPGAs provide efficient support for bitwise operations and can work with arbitraryprecision representation of numbers.This paper presents a new streaming architecture for running QNNs on FPGAs. The proposed architecture scales out better than alternatives, allowing us to take advantage of systems with multiple FPGAs. We also included support for skip connections, that are used in state-of-the art NNs, and shown that our architecture allows to add those connections almost for free. All this allowed us to implement an 18-layer ResNet for 224 × 224 images classification, achieving 57.5% top-1 accuracy.In addition, we implemented a full-sized quantized AlexNet. In contrast to previous works, we use 2-bit activations instead of 1bit ones, which improves AlexNet's top-1 accuracy from 41.8% to 51.03% for the ImageNet classification. Both AlexNet and ResNet can handle 1000-class real-time classification on an FPGA.Our implementation of ResNet-18 consumes 5× less power and is 4× slower for ImageNet, when compared to the same NN on the latest Nvidia GPUs. Smaller NNs, that fit a single FPGA, are running faster then on GPUs on small (32 × 32) inputs, while consuming up to 20× less energy and power.

Optimized Memory Allocation and Power Minimization for FPGA-Based Image Processing

“…Experiments show that our partitioning algorithms achieve higher efficiency than default synthesis strategies, except for frames of size 512 × 512 where the efficiency is unchanged. This is the case where default strategies perform equally well in terms of utilization since the image height and width are powers of 2 (refer back to Equation (13)). This confirms that modified partitioning strategies are required, according to requirements, in order to improve memory usage.…”

“…Within FPGA processing sub-systems, algorithms evolve from typical software-suitable representations into more hardware-friendly ones [6,11] which can fully exploit data parallelism [11] through application-specific hardware architectures [3], often substantially different from the traditional Von Neumann model, such as dataflow [12,13] or biologically inspired processing [14]. These heterogeneous architectures are customized for FPGA implementation not just for performance (e.g., by exploiting binary logarithmic arithmetic for efficient multiplication/division [15]), but also for power efficiency (e.g., by static/dynamic frequency scaling across parallel datapaths for reduced power consumption [16]).…”

Unrolling Ternary Neural Networks

“…Previous work [Baskin et al 2018;Chen et al 2017;Jouppi et al 2017;Kim et al 2017;Li et al 2017;Liang et al 2018;Meloni et al 2018;Moss et al 2017;Prost-Boucle et al 2017;Qiu et al 2016;Venkatesh et al 2017;Wang et al 2017;Zhang and Prasanna 2017] has mainly focused on general accelerators, implemented either as a sequence of instructions on fixed hardware, or accelerator platforms designed for linear algebra intensive computation. Performance comparisons with those in this paper are presented in Section 6.…”

“…There are two main approaches to compute the convolution. They are either passing in inputs over multiple cycles and storing intermediate results when computing the convolution, or buffering the pixels so that the entire set of inputs needed for the convolution is available simultaneously such as the approach in Baskin et al [Baskin et al 2018] and…”

“…This work implements a VGG-7 style network with ternary weights and activations. The approach is similar to Baskin et al [Baskin et al 2018] and Li et al [Li et al 2017] at a higher frequency of 250 MHz and using hand written register transfer language (RTL) as opposed to C-based high-level synthesis (HLS) tools. Their work is also the most similar to ours both in architecture and network choice, and they used an FPGA approximately 4× smaller than ours.…”