Block-Wise Separable Convolutions: An Alternative Way to Factorize Standard Convolutions

Abstract

In this paper, we introduce block-wise separable convolutions (BlkSConv) to replace the standard convolutions for compressing deep CNN models. First, BlkSConv expresses the standard convolutional kernel as an ordered set of block vectors each of which is a linear combination of fixed basis block vectors. Then it eliminates most basis block vectors and their corresponding coefficients to obtain an approximated convolutional kernel. Moreover, the proposed BlkSConv operation can be efficiently realized via a combination of pointwise and group-wise convolutions. Thus the constructed networks have smaller model size and fewer multiply-adds operations while keeping comparable prediction accuracy. We also develop a hyperparameter search framework based on principal component analysis (PCA) to determine a qualified hyperparameter setting of the block depth and number of basis block vectors. By this search framework, we construct networks which achieve nice prediction performance while simultaneously satisfying the constraints of model size and model efficiency. Our code, data, and models are available at https://github.com/yanjenhuang/blksconv.INDEX TERMS Convolutional neural network, Block-wise separable convolution, Network architecture search.