Evolving Deep DenseBlock Architecture Ensembles for Image Classification

Fielding, Ben; Zhang, Li

doi:10.3390/electronics9111880

Cited by 18 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the aforementioned A-BiLSTM architecture implemented in this research was shown to be highly effective, but with further experimentation with different layer and hyperparameter settings [24][25][26][27][28][29][30][31][32], additional improvements in performance could be made. Evolutionary algorithms [33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49] could also be exploited pertaining to the above parameter tuning as well as architecture generation processes. Moreover, it would also be beneficial to employ additional medical audio datasets to further evaluate model efficiency.…”

Section: Discussionmentioning

confidence: 99%

Deep Recurrent Neural Networks with Attention Mechanisms for Respiratory Anomaly Classification

Wall

Zhang

et al. 2021

2021 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

In recent years, a variety of deep learning techniques and methods have been adopted to provide AI solutions to issues within the medical field, with one specific area being audio-based classification of medical datasets. This research aims to create a novel deep learning architecture for this purpose, with a variety of different layer structures implemented for undertaking audio classification. Specifically, bidirectional Long Short-Term Memory (BiLSTM) and Gated Recurrent Units (GRU) networks in conjunction with an attention mechanism, are implemented in this research for chronic and non-chronic lung disease and COVID-19 diagnosis. We employ two audio datasets, i.e. the Respiratory Sound and the Coswara datasets, to evaluate the proposed model architectures pertaining to lung disease classification. The Respiratory Sound Database contains audio data with respect to lung conditions such as Chronic Obstructive Pulmonary Disease (COPD) and asthma, while the Coswara dataset contains coughing audio samples associated with COVID-19. After a comprehensive evaluation and experimentation process, as the most performant architecture, the proposed attention BiLSTM network (A-BiLSTM) achieves accuracy rates of 96.2% and 96.8% for the Respiratory Sound and the Coswara datasets, respectively. Our research indicates that the implementation of the BiLSTM and attention mechanism was effective in improving performance for undertaking audio classification with respect to various lung condition diagnoses.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep Recurrent Neural Networks with Attention Mechanisms for Respiratory Anomaly Classification

Wall

Zhang

et al. 2021

2021 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Fielding and Zhang [21] develop a PSO-based algorithm for image classification to optimize DenseBlock CNN architecture. The proposed method employs adaptive acceleration coefficients generated using cosine annealing learning rate schedule to overcome local optima traps.…”

Section: ) Control Of the Parametersmentioning

confidence: 99%

Exponential Particle Swarm Optimization for Global Optimization

et al. 2022

View full text Add to dashboard Cite

Nature-inspired metaheuristics have been extensively investigated to solve challenging optimization problems. Particle Swarm Optimization (PSO) is one of the most famous nature-inspired algorithms owing to its simplicity and ability to be used in a wide range of applications. This paper presents an extended PSO variant, namely, Exponential Particle Swarm Optimization (ExPSO). To effectively explore the whole search space, the proposed algorithm divides the swarm population into three equal subpopulations and employs a new search strategy based on an exponential function (permitting particles to make leaps in the search space) and an adapted control of the velocity range of each particle (to balance the exploration and exploitation search phases). The leaping strategy is integrated into the velocity equation and a new linear decreasing cognitive parameter (including a dynamic inertia weight strategy) is integrated into the proposed method. The developed algorithm allows large jumps at the beginning of the search, and then small jumps for further improvements in specific regions of the solution search space. Our variant approach, ExPSO, has been intensively tested through a comparison with eight other well-known heuristic search algorithms, over 29 benchmark problems, and real optimization engineering problems. The Wilcoxon signed-rank test and Friedman rank have been applied to analyze the search performance of the algorithms. The comparisons and statistical results show that the exponential search strategy significantly contributes to the search process and proves the superiority of ExPSO in terms of the convergence velocity and optimization accuracy.

show abstract

“…As a result, many PSO variants have been proposed to tackle the challenges. As an example, Fielding and Zhang [49] proposed a Swarm Optimised DenseBlock Architecture Ensemble (SODBAE) integrated with a PSO variant for image classification. The model was capable of devising CNN architectures with residual connections and dense connectivity to increase network diversity.…”

Section: Variants Of Particle Swarm Optimisationmentioning

confidence: 99%

An evolving ensemble model of multi-stream convolutional neural networks for human action recognition in still images

Slade

Zhang

et al. 2022

Neural Comput & Applic

Self Cite

View full text Add to dashboard Cite

Still image human action recognition (HAR) is a challenging problem owing to limited sources of information and large intra-class and small inter-class variations which requires highly discriminative features. Transfer learning offers the necessary capabilities in producing such features by preserving prior knowledge while learning new representations. However, optimally identifying dynamic numbers of re-trainable layers in the transfer learning process poses a challenge. In this study, we aim to automate the process of optimal configuration identification. Specifically, we propose a novel particle swarm optimisation (PSO) variant, denoted as EnvPSO, for optimal hyper-parameter selection in the transfer learning process with respect to HAR tasks with still images. It incorporates Gaussian fitness surface prediction and exponential search coefficients to overcome stagnation. It optimises the learning rate, batch size, and number of re-trained layers of a pre-trained convolutional neural network (CNN). To overcome bias of single optimised networks, an ensemble model with three optimised CNN streams is introduced. The first and second streams employ raw images and segmentation masks yielded by mask R-CNN as inputs, while the third stream fuses a pair of networks with raw image and saliency maps as inputs, respectively. The final prediction results are obtained by computing the average of class predictions from all three streams. By leveraging differences between learned representations within optimised streams, our ensemble model outperforms counterparts devised by PSO and other state-of-the-art methods for HAR. In addition, evaluated using diverse artificial landscape functions, EnvPSO performs better than other search methods with statistically significant difference in performance.

show abstract

Evolving Deep DenseBlock Architecture Ensembles for Image Classification

Cited by 18 publications

References 30 publications

Deep Recurrent Neural Networks with Attention Mechanisms for Respiratory Anomaly Classification

Deep Recurrent Neural Networks with Attention Mechanisms for Respiratory Anomaly Classification

Exponential Particle Swarm Optimization for Global Optimization

An evolving ensemble model of multi-stream convolutional neural networks for human action recognition in still images

Contact Info

Product

Resources

About