Improving random forest algorithm by selecting appropriate penalized method

Farhadi, Zari; Bevrani, Hossein; Feizi‐Derakhshi, Mohammad‐Reza

doi:10.1080/03610918.2022.2150779

Cited by 4 publications

(5 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a future work, we will use deep forest algorithm instead of 1D-CNNs as a learner instead of DRs in this study and RF in [4] and [5] . In addition, different structure of ensemble methods might be used instead of MLP, WA, and SA.…”

Section: Discussionmentioning

confidence: 99%

“…As can be seen in Figure (4), in the fusion phase, concatenation is used to merge DRs. In other words, in fusion phase, the extracted deep features of input data are concatenated.…”

Section: B the Aggregation Strategymentioning

confidence: 99%

“…The combination of the mentioned methods is compared with the non-shrinkage method. Figure (4) shows the structure of the MLP Ensemble model.…”

Section: B the Aggregation Strategymentioning

confidence: 99%

“…The other reason can be the numbers of learners. To reduce the error, Wang The authors of [4] introduced a novel hybrid model called RARTEN that combined EN and RF. The results revealed a prediction with MSE of 9.7350 and MAE of 2.0906 on Boston Housing Price dataset.…”

Section: Comparison With Past Studiesmentioning

confidence: 99%

“…In ML algorithms such as Random Forest (RF), increasing the number of trees causes overfitting, which creates problems in training phase as well as testing phase. This issue was investigated by Farhadi et al [4], [5]. In addition, in DL algorithms such as CNN, which have a layered structure, the depth of network increases by adding layer.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

ERDeR: The Combination of Statistical Shrinkage Methods and Ensemble Approaches to Improve the Performance of Deep Regression

Farhadi,

Feizi-Derakhshi,

Bevrani

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Ensembling is a powerful technique to obtain the most accurate results. In some cases, the large number of learners in ensemble learning mostly increases both computational load during the test phase and error rate. To solve this problem, in this paper we propose an Ensemble of Reduced Deep Regression (ERDeR) model, which is a combination of Deep Regressions (DRs), shrinkage methods, and ensemble approaches. The framework of the proposed model contains three phases. The first phase includes base regressions in which parallel DRs are used as learners. The role of these DRs is to extract features of input data and make prediction. In the second phase, to automatically reduce and select the most suitable DRs, shrinkage methods such as Least Absolute Shrinkage and Selection Operator (LASSO) and Elastic Net (EN) are employed. These models are compared with the non-shrinkage model. The last phase is ensemble phase, which consists of three different ensemble methods namely Multi-Layer Perceptron (MLP), Weighted Average (WA), and Simple Average (SA). These ensemble methods are used to aggregate the remaining learners from previous steps. Finally, the proposed model is applied to Monte Carlo simulation data and three real datasets including Boston House Price, Real Estate Valuation and Gold Price per Ounce. The results show that after applying the shrinkage methods the error rate is significantly reduced and the model accuracy is increased. Accordingly, the results of combining shrinkage methods and ensemble approaches not only decreased the computational load during test phase, but also increased the model accuracy.

show abstract

Section: Discussionmentioning

confidence: 99%

“…As can be seen in Figure (4), in the fusion phase, concatenation is used to merge DRs. In other words, in fusion phase, the extracted deep features of input data are concatenated.…”

Section: B the Aggregation Strategymentioning

confidence: 99%

“…The combination of the mentioned methods is compared with the non-shrinkage method. Figure (4) shows the structure of the MLP Ensemble model.…”

Section: B the Aggregation Strategymentioning

confidence: 99%

Section: Comparison With Past Studiesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations