An Ensemble Framework to Improve the Accuracy of Prediction Using Clustered Random-Forest and Shrinkage Methods

Abstract

Nowadays, in the topics related to prediction, in addition to increasing the accuracy of existing algorithms, the reduction of computational time is a challenging issue that has attracted much attention. Since the existing methods may not have enough efficiency and accuracy, we use a combination of machine-learning algorithms and statistical methods to solve this problem. Furthermore, we reduce the computational time in the testing model by automatically reducing the number of trees using penalized methods and ensembling the remaining trees. We call this efficient combinatorial method “ensemble of clustered and penalized random forest (ECAPRAF)”. This method consists of four fundamental parts. In the first part, k-means clustering is used to identify homogeneous subsets of data and assign them to similar groups. In the second part, a tree-based algorithm is used within each cluster as a predictor model; in this work, random forest is selected. In the next part, penalized methods are used to reduce the number of random-forest trees and remove high-variance trees from the proposed model. This increases model accuracy and decreases the computational time in the test phase. In the last part, the remaining trees within each cluster are combined. The results of the simulation and two real datasets based on the WRMSE criterion show that our proposed method has better performance than the traditional random forest by reducing approximately 12.75%, 11.82%, 12.93%, and 11.68% and selecting 99, 106, 113, and 118 trees for the ECAPRAF–EN algorithm.

Impact of Regional Mobility on Air Quality during COVID-19 Lockdown in Mississippi, USA Using Machine Learning

“…Hence, it would be difficult to determine from the observed data whether the changes in the concentrations (increase or decrease) are caused by weather conditions or by the traffic regulations implemented during the lockdown. By using machine learning models, one can subtract the weather component from the observation to obtain weather-normalized data that show the underlying causes of the change in the concentrations simulating a business-as-usual scenario (BAU) [18,[25][26][27][28][29][30][31][32]. Weather normalization can be achieved by using random forest (RF) regression models [41] via the 'randomForest' package in R [42].…”

“…Weather normalization can be achieved by using random forest (RF) regression models [41] via the 'randomForest' package in R [42]. "RF" regression is a type of ensemble learning method using many of what are known as "weak" predictors for building a forest of decision trees to obtain a good prediction accuracy [32].…”

“…Statistical and machine learning tools were used to analyze the air quality data to capture changes in the air quality, if any, due to the COVID-19 lockdown. Machine learning modeling was used to predict air pollutant concentrations during the lockdown period, by subtracting the influence of weather changes that might mask the true emission changes [18,[25][26][27][28][29][30][31][32].…”

Flight Delay Prediction Using Machine Learning: A Comparative Study of Ensemble Techniques

“…4) Random Forest Ensemble Method: Farhadi et al [27] explain that RF combines bagging with random feature selection to increase the diversity of DT models. It employs a voting mechanism to combine the predictions of multiple trees and enables fast training even with high-dimensional feature vectors.…”

ERDeR: The Combination of Statistical Shrinkage Methods and Ensemble Approaches to Improve the Performance of Deep Regression

“…In ML algorithms such as Random Forest (RF), increasing the number of trees causes overfitting, which creates problems in training phase as well as testing phase. This issue was investigated by Farhadi et al [4], [5]. In addition, in DL algorithms such as CNN, which have a layered structure, the depth of network increases by adding layer.…”