Quantized CNN: A Unified Approach to Accelerate and Compress Convolutional Networks

Abstract

We are witnessing an explosive development and widespread application of deep neural networks (DNNs) in various fields. However, DNN models, especially a convolutional neural network (CNN), usually involve massive parameters and are computationally expensive, making them extremely dependent on high-performance hardware. This prohibits their further extensions, e.g., applications on mobile devices. In this paper, we present a quantized CNN, a unified approach to accelerate and compress convolutional networks. Guided by minimizing the approximation error of individual layer's response, both fully connected and convolutional layers are carefully quantized. The inference computation can be effectively carried out on the quantized network, with much lower memory and storage consumption. Quantitative evaluation on two publicly available benchmarks demonstrates the promising performance of our approach: with comparable classification accuracy, it achieves 4 to $6 \times $ acceleration and 15 to $20\times $ compression. With our method, accurate image classification can even be directly carried out on mobile devices within 1 s.

DeepMUSIC: Multiple Signal Classification via Deep Learning

“…. , C y andW cy is the c y -th column vector ofW and C x = C y = 1024 is selected for f (14) (·). In (10), {f (i) (·)} i={2,5,8,11} represent the convolutional layers.…”

“…where d x × d y is the size of the convolutional kernel, and V x × V y are the size of the response of a convolutional layer.W vy,v k ∈ R Vx denotes the weights of the v y -th convolutional kernel, andX px ∈ R Vx is the input feature map at spatial position p x . Hence we define p x and p k as the 2-D spatial positions in the feature maps and convolutional kernels, respectively [13], [14]. In the proposed architecture, we use 256 filters, the first two of which are of size 5 × 5 and the remaining two have 3 × 3 filters.…”

Auto-tuning Neural Network Quantization Framework for Collaborative Inference Between the Cloud and Edge

“…In order to accelerate inference and compress the size of DNN models, many network quantization methods are proposed. Some studies focus on scalar and vector quantization [4,7], while others center on fixed-point quantization [18,19].…”

“…Recently, many neural network quantization methods have been proposed. Gong Y. et al [7] and Cheng J. et al [4] explored scalar and vector quantization methods for compressing DNNs. Zhou A. et al [18], Zhou S. et al [19] proposed fixed-point quantization methods.…”

Learning Student Networks via Feature Embedding

“…Recently, a variety of CNN compression methods have been proposed to tackle the aforementioned issues such as quantization [9], [10], weight and feature approximation [11], encoding [12], approximation [13], and pruning [14], [15]. Wherein, weight pruning based methods achieve the highest compression performance since there are considerable subtle weights in most of pre-trained CNNs.…”