C ir CNN

Abstract

Large-scale deep neural networks (DNNs) are both compute and memory intensive. As the size of DNNs continues to grow, it is critical to improve the energy efficiency and performance while maintaining accuracy. For DNNs, the model size is an important factor affecting performance, scalability and energy efficiency. Weight pruning achieves good compression ratios but suffers from three drawbacks: 1) the irregular network structure after pruning, which affects performance and throughput; 2) the increased training complexity; and 3) the lack of rigorous guarantee of compression ratio and inference accuracy.To overcome these limitations, this paper proposes CirCNN, a principled approach to represent weights and process neural networks using block-circulant matrices. CirCNN utilizes the Fast Fourier Transform (FFT)-based fast multiplication, simultaneously reducing the computational complexity (both in inference and training) from O(n 2 ) to O(n log n) and the storage complexity from O(n 2 ) to O(n), with negligible accuracy loss. Compared to other approaches, CirCNN is distinct due to its mathematical rigor: the DNNs based on CirCNN can converge to the same "effectiveness" as DNNs without compression. We propose the CirCNN architecture, a universal DNN inference engine that can be implemented in various hardware/software platforms with configurable network architecture (e.g., layer type, size, scales, etc.). In CirCNN architecture: 1) Due to the recursive property, FFT can be used as the key computing kernel, which ensures universal and small-footprint implementations.2) The compressed but regular network structure avoids the pitfalls of the network pruning and facilitates high performance and throughput with highly pipelined and parallel design. To demonstrate the performance and energy efficiency, we test Cir-CNN in FPGA, ASIC and embedded processors. Our results show that CirCNN architecture achieves very high energy efficiency and Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. MICRO-50, October 14-18, 2017 performance with a small hardware footprint. Based on the FPGA implementation and ASIC synthesis results, CirCNN achieves 6 -102X energy efficiency improvements compared with the best state-of-the-art results.

Run-Time Efficient RNN Compression for Inference on Edge Devices

“…We do an extensive comparison of HMD with 2 other compression techniques -model pruning and matrix factorization. Additionally, we also compared HMD with a structured matrix-based compression technique called block circular decomposition (BCD) [2,9]. BCD-compressed networks were able to recover the baseline accuracy for 2× -4× compression.…”

A new initialization method based on normed statistical spaces in deep networks

“…CirCNN networks. CirCNN implementation of neural networks in introduced in [2] is a promising approach to reduce number of parameters while preserving networks' topologies. This is done by replacing matrices and convolution kernels in a neural network by block circulant matrices and block circulant convolution kernels.…”

“…For a convolution layer with kernel W = (W ijk ), where the k and indices represent the input and output channels and the i and j indices represent 2D kernels, we say W is block-circulant if for each fixed i, j the resulting matrix (W ijk ) k is block-circulant. As suggested in [2], for a single layer the block size should be a constant, while one can choose different block sizes for different layers. Using CirCNN implementation can significantly reduce number of learnable parameters.…”

“…Entries in y are identically distributed with distribution Y with mean 0; 2. entries in ∂L/∂x are identically distributed with distribution ∂L/∂X with mean 0; 3. V(Y) = V(X ) and V(∂L/∂Y) = V(∂L/∂X ); 4. and E(||∆w|| 2 2 ) = E(||∂L/∂w|| 2 2 ). Note that conditions 1 to 3 are the adaptations of the Xavier/He initialization conditions to the layer type (6), while the condition 4 is a probabilistic description of a norm equality, which is explained at the end of this section.…”

Rank and run-time aware compression of NLP Applications

“…Structured matrices have shown significant potential for compression of NN (Sindhwani et al, 2015;Ding et al, 2017;Cheng et al, 2015;Thakker et al, 2020). Block circular compression is an extension of structured matrix based compression technique, converting every block in a matrix into structured matrix.…”