Exploration of Low Numeric Precision Deep Learning Inference Using Intel® FPGAs

Abstract

Convolutional neural networks (CNN) have been shown to maintain reasonable classification accuracy when quantized to lower precisions, however, quantizing to sub 8-bit activations and weights can result in classification accuracy falling below an acceptable threshold. Techniques exist for closing the accuracy gap of limited numeric precision networks typically by means of increasing computation. This results in a trade-off between throughput and accuracy and can be tailored for different networks through various combinations of activation and weight data widths. Customizable hardware architectures like FPGAs provide the opportunity for data width specific computation through unique logic configurations leading to highly optimized processing that is unattainable by full precision networks. Specifically, ternary and binary weighted networks offer an efficient method of inference for 2-bit and 1-bit data respectively. Most hardware architectures can take advantage of the memory storage and bandwidth savings that come along with a smaller datapath, but very few architectures can take full advantage of limited numeric precision at the computation level. In this paper, we present a hardware design for FPGAs that takes advantage of the bandwidth, memory, power, and computation savings of limited numerical precision data. We provide insights into the trade-offs between throughput and accuracy for various networks and how they map to our framework. Further, we show how limited numeric precision computation can be efficiently mapped onto FPGAs for both ternary and binary cases. Starting with Arria 10, we show a 2-bit activation and ternary weighted AlexNet running in hardware that achieves 3,700 images per second on the ImageNet dataset with a top-1 accuracy of 0.49. Using a hardware modeler designed for our low numeric precision framework we project performance most notably for a 55.5 TOPS Stratix 10 device running a modified ResNet-34 with only 3.7% accuracy degradation compared with single precision.

Performance-Efficiency Trade-off of Low-Precision Numerical Formats in Deep Neural Networks

“…For instance, the energy-delay-product of the posit EMAC with es = 0, on average, is 3× and 1.4× less than the energy-delay-product of the posit EMAC with es = 2 and es = 1, respectively. On the other hand, Fixed Float Posit the average performance of DNN inference with es = 1 for the posit EMAC among the five datasets and [5,7] bit-precision is 2% and 4% percent better than with es = 2 and es = 0, respectively. Thus, Deep Positron equipped with the posit (es = 1) EMAC has a better trade-off between energy-delay-product and accuracy for [5,7] bits.…”

“…On the other hand, Fixed Float Posit the average performance of DNN inference with es = 1 for the posit EMAC among the five datasets and [5,7] bit-precision is 2% and 4% percent better than with es = 2 and es = 0, respectively. Thus, Deep Positron equipped with the posit (es = 1) EMAC has a better trade-off between energy-delay-product and accuracy for [5,7] bits. For 8-bit, the results suggest that es = 1 is a better fit for energy-efficient applications and es = 2 for accuracy-dependent applications.…”

A review of the technological developments for interlocking at level crossing

“…The training of such models requires high computational power, however, once trained the network can work in realtime with even 15fps rate of processing power. The work in literature [61][62][63] mentions details about the training and computation power.…”

Convolution Accelerator Designs Using Fast Algorithms

“…At present, there are two solutions to accelerate the use of CNNs. One is to reduce the computational complexity of the neural network; many such methods have been proposed that maintain the accuracy, including quantification, tailoring, sparsity and fast convolution [7][8][9][10][11][12][13]. The other solution is to use a high-performance, low-power hardware accelerator.…”