Two‐Stage Bagging Pruning for Reducing the Ensemble Size and Improving the Classification Performance

Abstract

Ensemble methods, such as the traditional bagging algorithm, can usually improve the performance of a single classifier. However, they usually require large storage space as well as relatively time-consuming predictions. Many approaches were developed to reduce the ensemble size and improve the classification performance by pruning the traditional bagging algorithms. In this article, we proposed a two-stage strategy to prune the traditional bagging algorithm by combining two simple approaches: accuracy-based pruning (AP) and distance-based pruning (DP). These two methods, as well as their two combinations, “AP+DP” and “DP+AP” as the two-stage pruning strategy, were all examined. Comparing with the single pruning methods, we found that the two-stage pruning methods can furthermore reduce the ensemble size and improve the classification. “AP+DP” method generally performs better than the “DP+AP” method when using four base classifiers: decision tree, Gaussian naive Bayes, K-nearest neighbor, and logistic regression. Moreover, as compared to the traditional bagging, the two-stage method “AP+DP” improved the classification accuracy by 0.88%, 4.06%, 1.26%, and 0.96%, respectively, averaged over 28 datasets under the four base classifiers. It was also observed that “AP+DP” outperformed other three existing algorithms Brag, Nice, and TB assessed on 8 common datasets. In summary, the proposed two-stage pruning methods are simple and promising approaches, which can both reduce the ensemble size and improve the classification accuracy.

Validating machine learning models for the prediction of labour induction intervention using routine data: a registry-based retrospective cohort study at a tertiary hospital in northern Tanzania

“…Bagging or bootstrap aggregation method also uses ensemble learning to evolve ML models. 42 This algorithm is based on the hypothesis that combining multiple models together can often produce a much more powerful model. 43 Lreg is one of the simplest and most common ML algorithms that has been used often in low dimension data for binary classification problems.…”

UAV- and Random-Forest-AdaBoost (RFA)-Based Estimation of Rice Plant Traits

“…Although RF subtrees grow based on the training dataset through bagging, the average for all single trees is trained via parallel processing, which focuses only on a combination of strong prediction models [51], leading to data bias. Bagging, despite its widespread use in machine learning models to improve overfitting and reduce the variance of the data [52], suffers from the limitations of a small data quantity, data distribution and data quality issues.…”

Improvement in Tongue Color Image Analysis for Disease Identification Using Deep Learning Based Depthwise Separable Convolution Model

“…The classical bagging approach is comprised of 2 major modules like Bootstrap and Aggregation. Initially, the count of subsets is sampled in random fashion from actual training set with the help of bootstrap sampling principle [14] with substitution. Alternatively, bagging method is used in collecting the outputs of base methods with the help of voting principle for classification process.…”