HFOD: A hardware-friendly quantization method for object detection on embedded FPGAs

Abstract

There are two research hotspots for improving performance and energy efficiency of the inference phase of Convolutional neural networks (CNNs). The first one is model compression techniques while the second is hardware accelerator implementation. To overcome the incompatibility of algorithm optimization and hardware design, this paper proposes HFOD, a hardware-friendly quantization method for object detection on embedded FPGAs. We adopt a channel-wise, uniform quantization method to compress YOLOv3-Tiny model. Weights are quantized to 2-bit while activations are quantized to 8-bit for all convolutional layers. To achieve highly-efficient implementations on FPGA, we add batch normalization (BN) layer fusion in quantization process. A flexible, efficient convolutional unit structure is designed to utilize hardware-friendly quantization, and the accelerator is developed based on an automatic synthesis template. Experimental results show that the resources of FPGA in the proposed accelerator design contribute more computing performance compared with regular 8-bit/16-bit fixed point quantization. The model size and the activation size of the proposed network with 2-bit weights and 8-bit activations can be effectively reduced by 16× and 4× with a small amount of accuracy loss, respectively. Our HFOD method can achieve 90.6 GOPS on PYNQ-Z2 at 150 MHz, which is 1.4× faster and 2× better in power efficiency than peer FPGA implementation on the same platform.

Sample-wise dynamic precision quantization for neural network acceleration

“…Many accelerators with execution prediction are also proposed to reduce the computation of DNNs [8,9]. Quantization is a common step in DNNs deployment [10,11,12]. However, dynamic bit-width quantization has not been well studied.…”

Design and implementation of an efficient CNN accelerator for low-cost FPGAs

“…The designs proposed in [16,17,21] can deal with convolutions of several common kernel sizes, but it is still not applicable to convolutions of any kernel sizes. The authors in [22][23][24][25][26] adopt multiple computing engines to deal with the convolution with different kernel sizes for improving performance. However, this design costs too much hardware resources, which is not suitable for low-cost FPGAs.…”

“…Thus, without elaborately design, this mismatch between data and computation elements may lead to ultra-low utilization of FPGA resources, which is undesirable for low-cost FPGAs. For example, in [16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31], although the size of the CNN model is significanly reduced, the bit utilization of FPGA resources is very low. To solve the above problems, this brief proposes an efficient hardware accelerator for low-bit quantized lightweight CNN models and the contribution can be summarized as follows:…”