Evolving Deep Recurrent Neural Networks Using A New Variable-Length Genetic Algorithm

Viswambaran, Ramya Anasseriyil; Chen, Gang; Xue, Bing; Nekooei, Seyed Mohammad

doi:10.1109/cec48606.2020.9185851

Cited by 8 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FFNN optimization is proposed in [32], CNN optimization in [15]- [17,33] and RNN optimization in [33,34] is translated into single-objective using the scalarization method in [32] or Pareto Dominance (PD) approach in [16]. The scalarization method is very sensitive to the weighting of the objectives.…”

Section: B Supportive Combinationmentioning

confidence: 99%

“…It may require a large number of iterations to converge to a small part of the Pareto optimal front. Third, considering the way of DNN architectures building, there are two categories: layer stacking [15,17,32]- [34] and block stacking [16] approaches. The layer stacking approach is not preferred for complex classification problems.…”

Section: B Supportive Combinationmentioning

confidence: 99%

See 1 more Smart Citation

Evolutionary Optimization of Residual Neural Network Architectures for Modulation Classification

Perenda

Rajendran

Bovet

et al. 2022

IEEE Trans. Cogn. Commun. Netw.

View full text Add to dashboard Cite

Automatic modulation classification receives significant interest in the context of current and future wireless communication systems. Deep learning emerged as a powerful tool for modulation classification, as it allows for joint discriminative features learning and signal classification. However, the optimization of deep neural network architectures for modulation classification is a manual and time-consuming process that requires profound domain knowledge and much effort. Most state-of-the-art solutions focus mainly on classification accuracy, while optimization of network complexity is neglected. This paper presents a novel bi-objective memetic algorithm, BO-NSMA, to search optimal deep neural network architectures for modulation classification to maximize classification accuracy and minimize network complexity. The experiments show that BO-NSMA, with an initial population of six individuals and only ten generations, finds a deep neural network architecture that outperforms all human-crafted architectures. Furthermore, BO-NSMA discovered the first low-complexity Convolutional neural network architecture, which achieves slightly better performance than costly Recurrent neural network architectures, allowing a 2.9-fold reduction in network complexity with 1.43% performance improvement. Compared to counterparts from network architecture search, BO-NSMA finds the best architecture, which achieves up to 18.73% accuracy gain and up to an 82-fold reduction in network complexity. The results are validated using the Wilcoxon signed-rank test.

show abstract

Section: B Supportive Combinationmentioning

confidence: 99%

Section: B Supportive Combinationmentioning

confidence: 99%

Evolutionary Optimization of Residual Neural Network Architectures for Modulation Classification

Perenda

Rajendran

Bovet

et al. 2022

IEEE Trans. Cogn. Commun. Netw.

View full text Add to dashboard Cite

show abstract

“…2) Supportive combination: In this stream of work, GAs are used to optimize the connections and hyperparameters of the DNN architecture, while the weights are optimized using other algorithms such as the back-propagation [24]. FFNN optimization is proposed in [25], CNN optimization in [10,20,26,27] and RNN optimization in [26,28]. Mostly considered hyperparameters are the number of hidden layers, learning rate, type of optimizer, number of filters, layers' positions, and activation functions.…”

Section: Related Workmentioning

confidence: 99%

“…We can distinguish a few different research approaches within this stream. First, considering the network depth, we can separate them into two categories: fixed [10,27] and variable [20,26,28] network depth approaches. While the former might waste computational power in cases when the network depth is set to a value higher than optimal, the latter tries to find the optimal network depth and, thus, provides more computationally efficient candidate solutions.…”

Section: Related Workmentioning

confidence: 99%

Evolutionary Optimization of Residual Neural Network Architectures for Modulation Classification

Perenda¹,

Rajendran²,

Bovet³

et al. 2021

Preprint

View full text Add to dashboard Cite

Automatic Modulation Classification (AMC) receives significant interest in the context of current and future wireless communication systems. Deep learning emerged as a powerful AMC tool, as it allows for the joint learning of discriminative features, and signal classification. However, the optimization of Deep Neural Network (DNN) architectures for AMC is a manual and time-consuming process that requires profound domain knowledge and much effort. Moreover, most proposed solutions focus mainly on classification accuracy, while optimization of network complexity is neglected. In this paper, we propose a novel bi-objective memetic algorithm, BO-NSMA, to search optimal DNN architectures for AMC to maximize classification accuracy and minimize network complexity. We show that BO-NSMA, with a small initial population of six individuals and only ten generations, finds a DNN architecture that outperforms all human-crafted State-of-the-Art (SoA) models. BO-NSMA discovered the first low-complexity Convolutional Neural Network (CNN)-based model, which achieves slightly better performance than costly Recurrent Neural Network (RNN)-based approaches, allowing a 2.8-fold reduction in network complexity with 0.7% performance improvement. Compared to its counterparts from Network Architecture Search (NAS), BO-NSMA finds the best architecture, which achieves up to 18.24% accuracy gain and up to a 78.71-fold reduction in network complexity.

show abstract

“…Li et al provide a deep insight into the variable-length multiobjective optimization problems [23]. Viswambaran et al employ a GA with variable-length chromosomes to evolve deep recurrent neural networks (DRNNs) by using the variablelength encoding strategy to represent DRNNs of different depths [42]. The common characteristic of these graphs is that their feasible structures consist of an unfixed or unknown number of nodes or layers.…”

Section: Introductionmentioning

confidence: 99%

GAPORE: Boolean network inference using a genetic algorithm with novel polynomial representation and encoding scheme

Liu

Wang

Shi

et al. 2021

Knowledge-Based Systems

View full text Add to dashboard Cite

Evolving Deep Recurrent Neural Networks Using A New Variable-Length Genetic Algorithm

Cited by 8 publications

References 24 publications

Evolutionary Optimization of Residual Neural Network Architectures for Modulation Classification

Evolutionary Optimization of Residual Neural Network Architectures for Modulation Classification

Evolutionary Optimization of Residual Neural Network Architectures for Modulation Classification

GAPORE: Boolean network inference using a genetic algorithm with novel polynomial representation and encoding scheme

Contact Info

Product

Resources

About