Evolving CNN-LSTM Models for Time Series Prediction Using Enhanced Grey Wolf Optimizer

Xie, Hailun; Zhang, Li; Lim, Chee Peng

doi:10.1109/access.2020.3021527

Cited by 111 publications

(43 citation statements)

References 91 publications

(139 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Furthermore, the aforementioned A-BiLSTM architecture implemented in this research was shown to be highly effective, but with further experimentation with different layer and hyperparameter settings [24][25][26][27][28][29][30][31][32], additional improvements in performance could be made. Evolutionary algorithms [33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49] could also be exploited pertaining to the above parameter tuning as well as architecture generation processes. Moreover, it would also be beneficial to employ additional medical audio datasets to further evaluate model efficiency.…”

Section: Discussionmentioning

confidence: 99%

Deep Recurrent Neural Networks with Attention Mechanisms for Respiratory Anomaly Classification

Wall

Zhang

et al. 2021

2021 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

In recent years, a variety of deep learning techniques and methods have been adopted to provide AI solutions to issues within the medical field, with one specific area being audio-based classification of medical datasets. This research aims to create a novel deep learning architecture for this purpose, with a variety of different layer structures implemented for undertaking audio classification. Specifically, bidirectional Long Short-Term Memory (BiLSTM) and Gated Recurrent Units (GRU) networks in conjunction with an attention mechanism, are implemented in this research for chronic and non-chronic lung disease and COVID-19 diagnosis. We employ two audio datasets, i.e. the Respiratory Sound and the Coswara datasets, to evaluate the proposed model architectures pertaining to lung disease classification. The Respiratory Sound Database contains audio data with respect to lung conditions such as Chronic Obstructive Pulmonary Disease (COPD) and asthma, while the Coswara dataset contains coughing audio samples associated with COVID-19. After a comprehensive evaluation and experimentation process, as the most performant architecture, the proposed attention BiLSTM network (A-BiLSTM) achieves accuracy rates of 96.2% and 96.8% for the Respiratory Sound and the Coswara datasets, respectively. Our research indicates that the implementation of the BiLSTM and attention mechanism was effective in improving performance for undertaking audio classification with respect to various lung condition diagnoses.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep Recurrent Neural Networks with Attention Mechanisms for Respiratory Anomaly Classification

Wall

Zhang

et al. 2021

2021 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

show abstract

“…GWO has recently been used for other exploration problems. For instance, in [23], the authors used the GWO algorithm for evolving Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) networks for time series analysis. They showed that GWO can produce significantly better results than other meta-heuristic methods.…”

Section: Other Related Workmentioning

confidence: 99%

Efficient Design Space Exploration of OpenCL Kernels for FPGA Targets Using Black Box Optimization

Ghaffari

Savaria

2021

IEEE Access

View full text Add to dashboard Cite

Nowadays, many industries are in favor of using intelligent design-space exploration as opposed to brute-force analysis. In many applications, the design-space is defined by multiple variables and their interactions. Although brute-force analysis is very simple, it is rarely scalable when the number of variables in the system increases. With the rising complexity of hardware designs, more intelligent approaches are needed to explore the design options. This paper proposes using smart meta-heuristic search algorithms such as Grey Wolf Optimization (GWO) in conjunction with Bayesian Optimization (BO) to solve this problem. We show that we can further reduce the design effort using a surrogate model that is created based on a novel hybrid GWO-BO method. The surrogate model is a useful abstraction to detect functional and physical inter-dependencies in the system in order to accurately predict its performance (e.g. throughput or latency). We evaluate our methodology and show that it can produce competitive results in order to find the best design variables that maximize performance of the system. Finally, we compare our results with previous statistical and heuristic methods proposed in the literature and find that the proposed GWO-BO method always performs better than the other considered methods.

show abstract

“…The final results of trial and error method shows that three convolutional layers with a kernel size equal to 1 × 1 associated with three successive BiLSTM layers achieve better results according the highest the R 2 criteria. For the rest of optimization work, ES is employed to tune the critical hyperparameters of the proposed computing framework, specifically, the batch size, the number of units, the number of training epochs [49,50]. To tune these hyperparameters, Hyperactive library is employed on open source [39].…”

Section: Model Construction and Parameter Sittingsmentioning

confidence: 99%

PLS-CNN-BiLSTM: An End-to-End Algorithm-Based Savitzky–Golay Smoothing and Evolution Strategy for Load Forecasting

et al. 2020

View full text Add to dashboard Cite

This paper proposes an effective deep learning framework for Short-Term Load Forecasting (STLF) of multivariate time series. The proposed model consists of a hybrid Convolutional neural network-Bidirectional Long Short-Term Memory (CBiLSTM) based on the Evolution Strategy (ES) method and the Savitzky–Golay (SG) filter (SG-CBiLSTM). The adopted methodology incorporates the virtue of different prepossessing blocks to enhance the performance of the CBiLSTM model. In particular, a data-augmentation strategy is employed to synthetically improve the feature representation of the CBiLSTM model. The augmented data is forwarded to the Partial Least Square (PLS) method to select the most informative features above the predefined threshold. Next, the SG algorithm is computed for smoothing the load to enhance the learning capabilities of the underlying system. The structure of the SG-CBiLSTM for the ISO New England dataset is optimized using the ES technique. Finally, the CBiLSTM model generates output forecasts. The proposed approach demonstrates a remarkable improvement in the performance of the original CBiLSTM model. Furthermore, the experimental results strongly confirm the high effectiveness of the proposed SG-CBiLSTM model compared to the state-of-the-art techniques.

show abstract

Evolving CNN-LSTM Models for Time Series Prediction Using Enhanced Grey Wolf Optimizer

Cited by 111 publications

References 91 publications

Deep Recurrent Neural Networks with Attention Mechanisms for Respiratory Anomaly Classification

Deep Recurrent Neural Networks with Attention Mechanisms for Respiratory Anomaly Classification

Efficient Design Space Exploration of OpenCL Kernels for FPGA Targets Using Black Box Optimization

PLS-CNN-BiLSTM: An End-to-End Algorithm-Based Savitzky–Golay Smoothing and Evolution Strategy for Load Forecasting

Contact Info

Product

Resources

About