Recent advances in efficient computation of deep convolutional neural networks

Abstract

Deep neural networks have evolved remarkably over the past few years and they are currently the fundamental tools of many intelligent systems. At the same time, the computational complexity and resource consumption of these networks also continue to increase. This will pose a significant challenge to the deployment of such networks, especially in real-time applications or on resource-limited devices. Thus, network acceleration has become a hot topic within the deep learning community. As for hardware implementation of deep neural networks, a batch of accelerators based on FPGA/ASIC have been proposed in recent years. In this paper, we provide a comprehensive survey of recent advances in network acceleration, compression and accelerator design from both algorithm and hardware points of view. Specifically, we provide a thorough analysis of each of the following topics: network pruning, low-rank approximation, network quantization, teacher-student networks, compact network design and hardware accelerators. Finally, we will introduce and discuss a few possible future directions.

Comprehensive Evaluation of Supply Voltage Underscaling in FPGA on-Chip Memories

“…3) The Key Novelty of our Paper Related to NN Study: FPGAs are attractive devices to accelerate NN since they represent an intermediate point between the power and performance efficiency of ASICs and the programmability of CPUs and GPUs [67], [68], [69], [70]. One of the key components of FPGAs that directly impacts the performance of FPGAbased NNs is built-in BRAMs, due to the high-demand of NN computations for the parallel data access, as described in detail for recent FPGA-based accelerators in this survey paper [14].…”

Deep Neural Network Approximation for Custom Hardware

“…Cheng et al [25], Guo et al [49], Cheng et al [24] and Sze et al [136] surveyed algorithms for DNN compression and acceleration. Of these, Cheng et al [24] brie y evaluated system-level designs for FPGA implementation. Guo et al only surveyed quantisation methods; weight reduction was not mentioned.…”

Recognition and Classification of Incipient Cable Failures Based on Variational Mode Decomposition and a Convolutional Neural Network

“…This paper uses the CNN algorithm; as one of the deep learning models, its multilevel structure of local perception and value sharing can accommodate high-dimensional data more accurately and quickly. Due to the characteristics of self-learning, especially the results are stable in the field of image and signal recognition, and there is no additional feature engineering requirement [33].…”