Ensembles of Gradient Boosting Recurrent Neural Network for Time Series Data Prediction

Abstract

This work was supported in part by the Situation analysis and demonstration application of saffron cultivation and soil nutrients based on big data mining technology 201900014, Provincial Project in Data Fusion Analysis and Intelligent Regulation of Saffron Growth Environment Monitoring LGN19C130002 ABSTRACT Ensemble deep learning can combine strengths of neural network and ensemble learning and gradually becomes a new emerging research direction. However, the existing methods either lack theoretical support or demand large integrated models. To solve these problems, in this paper, Ensembles of Gradient Boosting Recurrent Neural Network (EGB-RNN) is proposed, which combines the gradient boosting ensemble framework with three types of recurrent neural network models, namely Minimal Gated Unit (MGU), Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM). RNN model is used as base learner to integrate an ensemble learner, through the way of gradient boosting. Meanwhile, for ensuring the ensemble model fit data better, Step Iteration Algorithm is designed to find an appropriate learning rate before models being integrated. Contrast trials are carried out on four time series data sets. Experimental results demonstrate that with the number of integration increasing, the performance of three types of EGB-RNN models tend to converge and the best EGB-RNN model and the best degree of ensemble vary with data sets. It is also shown in statistical results that the designed EGB-RNN models perform better than six baselines.

Intelligent Clinical Decision Support System for Managing COPD Patients

“…Some traditional machine learning techniques, such as random forest, SVM (support vector machine), Bayesian networks, and logistic regression, have been employed to improve predictive performance in identifying early clinical deterioration [ 27 ]. However, these traditional models are not optimized for handling the unique characteristics of time series data, such as autocorrelation, seasonality, and trend patterns [ 28 , 29 ].…”

“…Gradient boosting models are alternatives to specialized models, such as long short-term memory network (LSTM) and gated recurrent unit (GRU) [ 31 , 32 ]. Although these models are not ideal for time series forecasting, they are still generally better suited for handling sequential data compared to non-sequential algorithms (such as random forest, SVM, logistic regression, and naive Bayes) [ 29 ].…”

Stacked LSTM and Kernel-PCA-based Ensemble Learning for Cardiac Arrhythmia Classification

“…Stacking is one of the usual processes in combining models and training an ensemble model on the classification, enabling convenient experimentation and model comparisons. Even though Light Gradient Boosting is faster when compared with the performance it is equivalent [71][72][73]. The framework of the developed method is presented Fig.…”