Compressing Deep CNNs Using Basis Representation and Spectral Fine-Tuning

Abstract

We propose an efficient and straightforward method for compressing deep convolutional neural networks (CNNs) that uses basis filters to represent the convolutional layers, and optimizes the performance of the compressed network directly in the basis space. Specifically, any spatial convolution layer of the CNN can be replaced by two successive convolution layers: the first is a set of three-dimensional orthonormal basis filters, followed by a layer of one-dimensional filters that represents the original spatial filters in the basis space. We jointly fine-tune both the basis and the filter representation to directly mitigate any performance loss due to the truncation. Generality of the proposed approach is demonstrated by applying it to several well known deep CNN architectures and data sets for image classification and object detection. We also present the execution time and power usage at different compression levels on the Xavier Jetson AGX processor.

Simultaneous Learning and Compression for Convolution Neural Networks

“…To understand how we do it, consider the convolution layer of a CNN, whose filters are of size M × N × D × D, where N and M are number of input channels and output channels respectively and the spatial dimension of the filters is D × D. These filters are flattened and arranged as columns of the matrix H ∈ R M ×A , where A = N D 2 . Following [5,7] we compute the singular value decomposition of H. This lets us write it as H = USV T where U = F and SV T = W.…”

“…Low rank approximation compression methods [5,6,7,8] start with the assumption that the pretrained convolutions filters form a rank deficient matrix. They exploit this fact by representing the filters in a given convolution layer by the weighted linear combination of a set of basis filters [8,7].…”

“…Low rank approximation compression methods [5,6,7,8] start with the assumption that the pretrained convolutions filters form a rank deficient matrix. They exploit this fact by representing the filters in a given convolution layer by the weighted linear combination of a set of basis filters [8,7]. The network layer then operates by applying the basis filters first on the input followed by the weighted linear recombination of the channels of intermediate output.…”