Robust pruning for efficient CNNs

Ide, Hidenori; Kobayashi, Takumi; Watanabe, Kenji; Kurita, Takio

doi:10.1016/j.patrec.2020.03.034

Cited by 18 publications

(4 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Especially deep learning-based methods need high capacity computing resources due to the high number of parameters. In order to overcome this situation, some studies [22][23][24][25] should be carried out to reduce the number of parameters of deep learning architectures. In this study firstly, the state of the art deep learning models Googlenet [26], Alexnet [27], Vgg19 [28], resnet-50, and Resnet-101 [29] were used to achieve high accuracy in the classification of scenes in aerial images.…”

Section: Introductionmentioning

confidence: 99%

Classification of Scenes in Aerial Images with Deep Learning Models

İni̇k

2023

Türk Doğa Ve Fen Dergisi

View full text Add to dashboard Cite

Automatic classification of aerial images has become one of the topics studied in recent years. Especially for the use of drones in different fields such as agricultural applications, smart city applications, surveillance and security applications, it is necessary to automatically classify the images obtained with the camera during autonomous mission execution. For this purpose, researchers have created new data sets and some computer vision methods have been developed to achieve high accuracy. However, in addition to increasing the accuracy of the developed methods, the computational complexity should also be reduced. Because the methods to be used in devices such as drones where energy consumption is important should have low computational complexity. In this study, firstly, five different state-of-art deep learning models were used to obtain high accuracy values in the classification of aerial images. Among these models, the VGG19 model achieved the highest accuracy with 94.21%. In the second part of the study, the parameters of this model were analyzed and the model was reconstructed. The number of 143.6 million parameters of the VGG19 model was reduced to 34 million. The accuracy of the model obtained by reducing the number of parameters is 93.56% on the same test data. Thus, despite the 66.5% decrease in the parameter ratio, there was only a 0.7% decrease in the accuracy value. When compared to previous studies, the results show improved performance.

show abstract

Section: Introductionmentioning

confidence: 99%

Classification of Scenes in Aerial Images with Deep Learning Models

İni̇k

2023

Türk Doğa Ve Fen Dergisi

View full text Add to dashboard Cite

show abstract

“…The trade off between accuracy and number of parameters occur for pursuing efficient CNN. Several researches have introduced various techniques include pruning, regularization, and efficient architectures to produce CNN with low number of parameters without sacrificing accuracy in diverse topics (Yudistira, Widodo, & Rahayudi, 2020) (Hakim, Kavitha, Yudistira, & Kurita, 2021) (Mitsuno & Kurita, 2021) (Ide, Kobayashi, Watanabe, & Kurita, 2020). In this research, we choose three different CNN architectures that will be evaluated to achieve the best results: ResNet-50, Inception-V3, and EfficientNet-B7.…”

Section: Introductionmentioning

confidence: 99%

Efficient CNN for high-resolution remote sensing imagery understanding

Yudistira

Santoso

2023

Multimed Tools Appl

View full text Add to dashboard Cite

Remote Sensing is one of the relatively complex problems in Machine Learning because of spatial patterns and intricate geometric structures of the data that make semantic understanding meaning essential in the remote sensing community. CNN is one of the Machine Learning methods often used in Remote Sensing problems. However, high-resolution aerial view classification often leverages large-scale data with a huge number of parameters of the CNN model. That large number of parameters makes it hard to be applied to remote imaging peripherals because it requires high capacity of storage and memory. We propose a training framework to solve that problem that produces a model with minimum parameters without sacrificing accuracy. In this study, the three CNN architectures are used to be compared: ResNet, Inception, and EfficientNet. Furthermore, the most efficient CNN backbone then leverages Freezing layers, adding Weighted Loss and Sparse Regularization during training and then Pruning after training. Using this method on the AID Dataset, the best results are achieved by EfficientNet-B0 with Freeze 2 Layer, INS Weighted Loss, Sparse regularization with λ = 0.001, and Global Unstructured Conv2D Pruning with an accuracy of 95.86% on test data with total parameters of 2,463,501. This study proves that Weighted Loss and Sparse Regularization can help the model to improve accuracy while Pruning enhances efficiency by cutting the model parameters into a few.

show abstract

“…They demonstrated that the feature maps generated at various layers of this model have redundancy and they proposed a singular value decomposition-based framework to reduce redundant feature maps [25]. Ide H. et al proposed a pruning method based on a novel criterion to measure the redundancy of the CNN parameters through the classification loss [26]. Recently, the authors in [27] proposed a kernel-based weight decorrelation approach to regularize the CNNs for better model generalization.…”

Section: Introductionmentioning

confidence: 99%

KRR-CNN: kernels redundancy reduction in convolutional neural networks

Hssayni

Joudar

Ettaouil

2021

Neural Comput & Applic

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) are a promising tool for solving real-world problems. However, successful CNNs often require a large number of parameters, which leads to a significant amount of memory and a higher computational cost. This may produce some undesirable phenomena, notably the overfitting. Indeed, in CNNs, many kernels are usually redundant and can be eliminated from the network while preserving the performance. In this work, we propose a new optimization model for kernels redundancy reduction in CNN named KRR-CNN. It consists of minimization and optimization phases. In the first one, a dataset is used to train a specific CNN generating a learned CNN with optimal parameters. These later are combined with a decision optimization model to reduce kernels that have not contributed to the first task. The optimization phase is carried out by the evolutionary genetic algorithm. Efficiency of KRR-CNN has been demonstrated by several experiments. In fact, the suggested model allows reducing the kernels redundancy and improving the classification performance comparable to the state-of-the-art CNNs.

show abstract

Robust pruning for efficient CNNs

Cited by 18 publications

References 2 publications

Classification of Scenes in Aerial Images with Deep Learning Models

Classification of Scenes in Aerial Images with Deep Learning Models

Efficient CNN for high-resolution remote sensing imagery understanding

KRR-CNN: kernels redundancy reduction in convolutional neural networks

Contact Info

Product

Resources

About