Holistic Filter Pruning for Efficient Deep Neural Networks

Abstract

Deep neural networks (DNNs) are usually overparameterized to increase the likelihood of getting adequate initial weights by random initialization. Consequently, trained DNNs have many redundancies which can be pruned from the model to reduce complexity and improve the ability to generalize. Structural sparsity, as achieved by filter pruning, directly reduces the tensor sizes of weights and activations and is thus particularly effective for reducing complexity. We propose Holistic Filter Pruning (HFP), a novel approach for common DNN training that is easy to implement and enables to specify accurate pruning rates for the number of both parameters and multiplications. After each forward pass, the current model complexity is calculated and compared to the desired target size. By gradient descent, a global solution can be found that allocates the pruning budget over the individual layers such that the desired target size is fulfilled. In various experiments, we give insights into the training and achieve state-of-the-art performance on CIFAR-10 and ImageNet (HFP prunes 60% of the multiplications of ResNet-50 on ImageNet with no significant loss in the accuracy). We believe our simple and powerful pruning approach to constitute a valuable contribution for users of DNNs in low-cost applications.

Target Capacity Filter Pruning Method for Optimized Inference Time Based on YOLOv5 in Embedded Systems

“…Pruning is classified as either unstructured/weight pruning [14] or structured pruning [15]- [22] according to the method of determining the importance of parameters and removing them. Unstructured pruning [14] removes parameters by determining the importance of each parameter according to the saliency score of each algorithm.…”

“…Unstructured pruning also has the disadvantage of requiring specific library and hardware support for a sparse matrix. In contrast, structured pruning [15]- [22] judges the importance of network connections according to the saliency score of each algorithm based on larger units such as channels. Structured pruning does not have a sparse matrix so that an existing library can be used, and memory usage can be reduced without requiring additional hardware.…”

“…After training the network to figure out the importance of the parameters, unimportant parameters are removed according to their saliency scores by each algorithm. Moreover, pruning is divided into unstructured pruning [14] and structured pruning [15]- [22]. Unstructured pruning [14] judges the importance of each parameter individually and removes unnecessary parameters to obtain a sparse convolution structure.…”

“…Unstructured pruning [14] judges the importance of each parameter individually and removes unnecessary parameters to obtain a sparse convolution structure. Structured pruning [15]- [22] removes unnecessary parameters by judging the importance of a bundle of several parameters. In this paper, by using structured pruning, YOLOv5 (an object detection network), is light-weighted to generate a network with a significantly reduced number of parameters and amount of computation.…”

“… Inspired by the holistic filter pruning(HFP) method proposed in [15], a network with a target number of parameters and FLOPs can be generated using the proposed target capacity filter pruning (TCFP). This was applied to the latest object detection network, YOLOv5, and the performance was evaluated on an edge device, NVIDIA Jetson Xavier NX.…”