Providing clear pruning threshold: A novel CNN pruning method via L ₀ regularisation

Abstract

Network pruning is a significant way to improve the practicability of convolution neural networks (CNNs) by removing the redundant structure of the network model. However, in most of the existing network pruning methods l1 or l2 regularisation is applied to parameter matrices and the manual selection of pruning threshold is difficult and labor‐intensive. A novel CNNs network pruning method via l0 regularisation is proposed, which adopts l0 regularisation to expand the saliency gap between neurons. A half‐quadratic splitting (HQS) based iterative algorithm is put forward to calculate the approximation solution of l0 regularisation, which makes the joint optimisation problem of regularisation term and training loss function can be solved by various gradient‐based algorithms. Meanwhile, a hyperparameters selection method is designed to make most of the hyperparameters in the algorithm can be determined by examining the pre‐trained model. The results of experiments on MNIST, Fashion‐MNIST and CIFAR100 show that the proposed method can provide a much clearer pruning threshold by widening the saliency gap, and achieve a similar or even better compression performance, compared with the state‐of‐the‐art studies.

A novel MobileNet with selective depth multiplier to compromise complexity and accuracy

“…They achieved higher accuracy than the previous method. Furthermore Li and Xu [6] proposed a new pruning method using l 0 regularization, focus on model compression rather than accuracy improvement.…”