KRR-CNN: kernels redundancy reduction in convolutional neural networks

Abstract

Convolutional neural networks (CNNs) are a promising tool for solving real-world problems. However, successful CNNs often require a large number of parameters, which leads to a significant amount of memory and a higher computational cost. This may produce some undesirable phenomena, notably the overfitting. Indeed, in CNNs, many kernels are usually redundant and can be eliminated from the network while preserving the performance. In this work, we propose a new optimization model for kernels redundancy reduction in CNN named KRR-CNN. It consists of minimization and optimization phases. In the first one, a dataset is used to train a specific CNN generating a learned CNN with optimal parameters. These later are combined with a decision optimization model to reduce kernels that have not contributed to the first task. The optimization phase is carried out by the evolutionary genetic algorithm. Efficiency of KRR-CNN has been demonstrated by several experiments. In fact, the suggested model allows reducing the kernels redundancy and improving the classification performance comparable to the state-of-the-art CNNs.

A deep learning framework for time series classification using normal cloud representation and convolutional neural network optimization

“…The use of $$$ CNN $$$ for classification task is generally applied using default settings, i.e without a study of the redundancy in these models. Moreover, the distribution of redundancy in CNNs is not clearly described 31 . In this paper, the use of CNNs for TSC is reinforced by a combinatorial optimization approach.…”

“…Its efficiency is due to its inspiration by the process of developing a biological system based on Darwin's theory of evolution 32 . This is one of the most common algorithms for combinatorial optimization problems related to other meta‐heuristics 31,33 . Based on a group of random solutions called individuals, two parents are specifically selected to recreate a new generation.…”

“…Moreover, the distribution of redundancy in CNNs is not clearly described. 31 In this paper, the use of CNNs for TSC is reinforced by a combinatorial optimization approach. The suggested model presents a new vision of the redundancy localization and reduction by defining a set of decision variables.…”

“…32 This is one of the most common algorithms for combinatorial optimization problems related to other meta-heuristics. 31,33 Based on a group of random solutions called individuals, two parents are specifically selected to recreate a new generation. The reproductive task consists of crossovers and mutations, the first one increases the population diversity, and the second riches the intensification.…”

Classification on Unsupervised Deep Hashing With Pseudo Labels Using Support Vector Machine for Scalable Image Retrieval

“…The muddle codes greatly minimize the need for extra room to improve mathematical abilities [8], [9]. As of late, many hashing methods [10][11][12][13] have carried out impressive executions.…”