Adaptive CNN filter pruning using global importance metric

Mondal, Milton; Das, Bishshoy; Roy, Sumantra Dutta; Singh, Pushpendra; Lall, Brejesh; Joshi, Sachindra

doi:10.1016/j.cviu.2022.103511

Cited by 19 publications

(8 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The choice of those criteria and partition rate is motivated by the results in Table 1 and Table 3 for VGGNet and ResNet models pruning on the CIFAR-10 dataset. Table 5 demonstrate the superiority of our method with several other state-of-the-art pruning methods such as SFP [13], FPGM [15], MetaPruning [51], MFP [16], LFC [52], FuPruner [53], GFI-AP [25] etc. when pruning ResNet on ImageNet dataset.…”

Section: Resnet On Imagenetmentioning

confidence: 85%

“…The weight or connection pruning eliminates the unimportant connections from the network resulting in an unstructured sparse network [10][11][12]28]. The filter or channel pruning eliminates the entire filter or channel retaining the original structure of the network [13,14,16,[24][25][26][27]29]. In contrast to filter pruning, sparse networks after weight pruning require specialized hardware or software in order to achieve practical acceleration.…”

Section: Related Workmentioning

confidence: 99%

“…Also, Bayesian optimization-based variable rate one-shot filter pruning is recently presented by Kim et al [29], in which optimal pruning rates for each layer are automatically identified. Recently, GFI-AP [25] extensively studies the filter importance criteria for each layer and proposes that variable rate pruning across convolution layers outperforms fixed rate filter pruning. Instead of selecting pruning filters using magnitude or similarity criteria, Lin et al [38] formulated the information preserving of pretrained CNN filters as a matrix sketch problem, but they still use the fixed rate pruning for all pruning layers in the network.…”

Section: Filter Pruningmentioning

confidence: 99%

“…Pruning is a promising approach for neural network compression that can be divided into two main categories, weight pruning (e.g., [10,11]), and filter pruning (e.g., [13,24,25]). Weight pruning removes the individual entries in the filter or connection in fully connected layers and the resulting network is unstructured and sparse.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Magnitude and Similarity Based Variable Rate Filter Pruning for Efficient Convolution Neural Networks

Ghimire

Kim

2022

Applied Sciences

View full text Add to dashboard Cite

The superior performance of the recent deep learning models comes at the cost of a significant increase in computational complexity, memory use, and power consumption. Filter pruning is one of the effective neural network compression techniques suitable for model deployment in modern low-power edge devices. In this paper, we propose a loss-aware filter Magnitude and Similarity based Variable rate Filter Pruning (MSVFP) technique. We studied several filter selection criteria based on filter magnitude and similarity among filters within a convolution layer, and based on the assumption that the sensitivity of each layer throughout the network is different, unlike conventional fixed rate pruning methods, our algorithm using loss-aware filter selection criteria automatically finds the suitable pruning rate for each layer throughout the network. In addition, the proposed algorithm adapts two different filter selection criteria to remove weak filters as well as redundant filters based on filter magnitude and filter similarity score respectively. Finally, the iterative filter pruning and retraining approach are used to maintain the accuracy of the network during pruning to its target float point operations (FLOPs) reduction rate. In the proposed algorithm, a small number of retraining steps are sufficient during iterative pruning to prevent an abrupt drop in the accuracy of the network. Experiments with commonly used VGGNet and ResNet models on CIFAR-10 and ImageNet benchmark show the superiority of the proposed method over the existing methods in the literature. Notably, VGG-16, ResNet-56, and ResNet-110 models on the CIFAR-10 dataset even improved the original accuracy with more than 50% reduction in network FLOPs. Additionally, the ResNet-50 model on the ImageNet dataset reduces model FLOPs by more than 42% with a negligible drop in the original accuracy.

show abstract

Section: Resnet On Imagenetmentioning

confidence: 85%

Section: Related Workmentioning

confidence: 99%

Section: Filter Pruningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Magnitude and Similarity Based Variable Rate Filter Pruning for Efficient Convolution Neural Networks

Ghimire

Kim

2022

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Especially deep learning-based methods need high capacity computing resources due to the high number of parameters. In order to overcome this situation, some studies [22][23][24][25] should be carried out to reduce the number of parameters of deep learning architectures. In this study firstly, the state of the art deep learning models Googlenet [26], Alexnet [27], Vgg19 [28], resnet-50, and Resnet-101 [29] were used to achieve high accuracy in the classification of scenes in aerial images.…”

Section: Introductionmentioning

confidence: 99%

Classification of Scenes in Aerial Images with Deep Learning Models

İni̇k

2023

Türk Doğa Ve Fen Dergisi

View full text Add to dashboard Cite

Automatic classification of aerial images has become one of the topics studied in recent years. Especially for the use of drones in different fields such as agricultural applications, smart city applications, surveillance and security applications, it is necessary to automatically classify the images obtained with the camera during autonomous mission execution. For this purpose, researchers have created new data sets and some computer vision methods have been developed to achieve high accuracy. However, in addition to increasing the accuracy of the developed methods, the computational complexity should also be reduced. Because the methods to be used in devices such as drones where energy consumption is important should have low computational complexity. In this study, firstly, five different state-of-art deep learning models were used to obtain high accuracy values in the classification of aerial images. Among these models, the VGG19 model achieved the highest accuracy with 94.21%. In the second part of the study, the parameters of this model were analyzed and the model was reconstructed. The number of 143.6 million parameters of the VGG19 model was reduced to 34 million. The accuracy of the model obtained by reducing the number of parameters is 93.56% on the same test data. Thus, despite the 66.5% decrease in the parameter ratio, there was only a 0.7% decrease in the accuracy value. When compared to previous studies, the results show improved performance.

show abstract

Green AI‐Driven Concept for the Development of Cost‐Effective and Energy‐Efficient Deep Learning Method: Application in the Detection ofEimeriaParasites as a Case Study

Acmali,

Ortakci,

Seker

2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Although large‐scale pretrained convolutinal neural networks (CNN) models have shown impressive transfer learning capabilities, they come with drawbacks such as high energy consumption and computational cost due to their potential redundant parameters. This study presents an innovative weight‐level pruning technique that mitigates the challenges of overparameterization, and subsequently minimizes the electricity usage of such large deep learning models. The method focuses on removing redundant parameters while upholding model accuracy. This methodology is applied to classify Eimeria species parasites from fowls and rabbits. By leveraging a set of 27 pretrained CNN models with a number of parameters between 3.0M and 118.5M, the framework has identified a 4.8M‐parameter model with the highest accuracy for both animals. The model is then subjected to a systematic pruning process, resulting in an 8% reduction in parameters and a 421M reduction in floating point operations while maintaining the same classification accuracy for both fowls and rabbits. Furthermore, unlike the existing literature where two separate models are created for rabbits and fowls, this article presents a combined model with 17 classes. This approach has resulted in a CNN model with nearly 50% reduced parameter size while retaining the same accuracy of over 90%.

show abstract

Adaptive CNN filter pruning using global importance metric

Cited by 19 publications

References 24 publications

Magnitude and Similarity Based Variable Rate Filter Pruning for Efficient Convolution Neural Networks

Magnitude and Similarity Based Variable Rate Filter Pruning for Efficient Convolution Neural Networks

Classification of Scenes in Aerial Images with Deep Learning Models

Green AI‐Driven Concept for the Development of Cost‐Effective and Energy‐Efficient Deep Learning Method: Application in the Detection ofEimeriaParasites as a Case Study

Contact Info

Product

Resources

About