Evolving Deep Architecture Generation with Residual Connections for Image Classification Using Particle Swarm Optimization

Abstract

Automated deep neural architecture generation has gained increasing attention. However, exiting studies either optimize important design choices, without taking advantage of modern strategies such as residual/dense connections, or they optimize residual/dense networks but reduce search space by eliminating fine-grained network setting choices. To address the aforementioned weaknesses, we propose a novel particle swarm optimization (PSO)-based deep architecture generation algorithm, to devise deep networks with residual connections, whilst performing a thorough search which optimizes important design choices. A PSO variant is proposed which incorporates a new encoding scheme and a new search mechanism guided by non-uniformly randomly selected neighboring and global promising solutions for the search of optimal architectures. Specifically, the proposed encoding scheme is able to describe convolutional neural network architecture configurations with residual connections. Evaluated using benchmark datasets, the proposed model outperforms existing state-of-the-art methods for architecture generation. Owing to the guidance of diverse non-uniformly selected neighboring promising solutions in combination with the swarm leader at fine-grained and global levels, the proposed model produces a rich assortment of residual architectures with great diversity. Our devised networks show better capabilities in tackling vanishing gradients with up to 4.34% improvement of mean accuracy in comparison with those of existing studies.

A Deep Ensemble Neural Network with Attention Mechanisms for Lung Abnormality Classification Using Audio Inputs

“…Such optimized settings equip the network with distinctive learning behaviors and help prevent the network from overfitting. In addition, the PSO algorithm is widely adopted for solving diverse optimization problems, such as ensemble classifier reduction [ 38 ], feature selection [ 39 ], deep architecture generation [ 40 ], hyper-parameter identification [ 41 ], and job scheduling [ 42 ]. In comparison with other swarm intelligence algorithms, such as the Firefly Algorithm and Simulated Annealing, it searches for the most optimal solution by following both personal and global best experiences.…”

Neural Inference Search for Multiloss Segmentation Models

“…A swarm intelligence algorithm with crossover operators based on sine, cosine and tanh functions was also utilized by Zhang et al [25] for bidirectional LSTM network generation pertaining to video action recognition. Lawrence et al [26] developed a PSO variant with a residual group-based encoding mechanism for residual CNN generation. PSO with population aggregation measurement was integrated with generative adversarial networks (GANs) for facial image generation in Zhang and Zhao [27].…”

Parçacık Sürü Optimizasyonu Yoluyla Geliştirilen Doğrusal Bir Sınıflandırıcının Analizi

“…A way to minimize the cost function is to utilize meta-heuristic algorithms. There are lots of works that Meta-heuristics have been used for solving classification problems (Athira Lekshmi et al, 2018;De Falco et al, 2007;Lawrence et al, 2021;M. G. Omran et al, 2002;M.…”