An ensemble based on neural networks with random weights for online data stream regression

Abstract

Most information sources in the current technological world are generating data sequentially and rapidly, in the form of data streams. The evolving nature of processes may often cause changes in data distribution, also known as concept drift, which is difficult to detect and causes loss of accuracy in supervised learning algorithms. As a consequence, online machine learning algorithms that are able to update actively according to possible changes in the data distribution are required. Although many strategies have been developed to tackle this problem, most of them are designed for classification problems. Therefore, in the domain of regression problems, there is a need for the development of accurate algorithms with dynamic updating mechanisms that can operate in a computational time compatible with today's demanding market. In this article, the authors propose a new bagging ensemble approach based on neural network with random weights for online data stream regression. The proposed method improves the data prediction accuracy as well as minimises the required computational time compared to a recent algorithm for online data stream regression from literature. The experiments are carried out using four synthetic datasets to evaluate the algorithm's response to concept drift, along with four benchmark datasets from different industries. The results indicate improvement in data prediction accuracy, effectiveness in handling concept drift, and much faster updating times compared to the existing available approach. Additionally, the use of design of experiments as an effective tool for hyperparameter tuning is demonstrated. Keywords Ensembles Á Data stream regression Á Neural networks with random weights Á Hyperparameter adjustment Communicated by V. Loia.

Trust Assessment on Streaming Data: A Real Time Predictive Approach

“…Basic regression models are SGD-Regressor and Passive-Aggressive-Regressor 3 . Pruning strategy is the worst first [3].…”

“…Data is accessed only once and then discarded to limit memory and storage space usages [9]. -Batch-by-Batch update has better stability than instance-by-instance [3,8] and is less sensitive to inaccurate data [3]. -Implicit method is more suitable than the explicit one (such as concept detection) in noisy data streams [20], because the latter may cause too many false alarms [8,20].…”

“…As illustrated in Table 1, AddExp [19] and B-NNRW [3] are the methods that meet our criteria. AddExp uses a loss bound to obtain the error model, and adjusts the expert's weights according to their actual losses (M1).…”

Ensemble Learning of Hybrid Acoustic Features for Speech Emotion Recognition

“…It improves the overall predictive performance, decreases the risk of obtaining a local minimum and provides a better fit to the data space by combining the predictions of several weak learners into a strong learning algorithm. In this study, we applied bagging and boosting machine learning because they are widely used effective approaches for constructing ensemble learning algorithms [76][77][78][79]. Bagging is a technique that utilizes bootstrap sampling to reduce the variance of a decision tree and improve the accuracy of a learning algorithm by creating a collection of learning algorithms that are learned in parallel [79][80][81].…”

Ensemble Algorithms and Their Applications

“…For our experiments, the evolutionary operators, a) ranking selection, b) uniform crossover, and c) random mutation were used [76,77]. In addition, we applied elite strategy selection, while a probability of crossover P c and mutation P m was assigned to each population.…”