Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning

Abstract

Deep convolutional neural networks (CNNs) are indispensable to state-of-the-art computer vision algorithms. However, they are still rarely deployed on battery-powered mobile devices, such as smartphones and wearable gadgets, where vision algorithms can enable many revolutionary real-world applications. The key limiting factor is the high energy consumption of CNN processing due to its high computational complexity. While there are many previous efforts that try to reduce the CNN model size or the amount of computation, we find that they do not necessarily result in lower energy consumption. Therefore, these targets do not serve as a good metric for energy cost estimation.To close the gap between CNN design and energy consumption optimization, we propose an energy-aware pruning algorithm for CNNs that directly uses the energy consumption of a CNN to guide the pruning process. The energy estimation methodology uses parameters extrapolated from actual hardware measurements. The proposed layerby-layer pruning algorithm also prunes more aggressively than previously proposed pruning methods by minimizing the error in the output feature maps instead of the filter weights. For each layer, the weights are first pruned and then locally fine-tuned with a closed-form least-square solution to quickly restore the accuracy. After all layers are pruned, the entire network is globally fine-tuned using backpropagation. With the proposed pruning method, the energy consumption of AlexNet and GoogLeNet is reduced by 3.7× and 1.6×, respectively, with less than 1% top-5 accuracy loss. We also show that reducing the number of target classes in AlexNet greatly decreases the number of weights, but has a limited impact on energy consumption.

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

“…Finally, there are various methods to reduce the weights in a DNN (e.g., network pruning in Section VII-B2). Table IV shows another example of these DNN model metrics, by comparing sparse DNNs pruned using [142] to dense DNNs.…”

“…35 [80]. Rather than using the number of weights and MAC operations as proxies for energy, the pruning of the weights can be directly driven by energy itself [142]. An energy evaluation method can be used to estimate the DNN energy that accounts for the data movement from different levels of the memory hierarchy, the number of MACs, and the data sparsity as shown in Fig.…”

Pre-defined Sparsity for Low-Complexity Convolutional Neural Networks

“…It is observed that the CSR 8 and CSR 16 yield lower total storage when the original matrix is at most 82% and 73% dense, respectively. Furthermore, at 62% density, CSR 8 and CSR 16 yield lower total storage compared to CSR by 23% and 16%, respectively § . This is equivalent to 60.04% and 39.86% reduction in the overhead of storing auxiliary vectors for the CSR 8 and CSR 16 compared to the CSR format, respectively.…”

PreVIous: A Methodology for Prediction of Visual Inference Performance on IoT Devices

“…As a first step for modeling the performance of embedded CNNs, recent studies have carried out systematic benchmarking on several hardware systems [6,[27][28][29]. Gaining in specificity, an energy estimation methodology for CNN accelerators has been introduced in [30,31]. Power monitor tools available on GPU-based platforms have also been employed to measure and model energy consumption [32][33][34].…”