Application of adaptive boosting (AdaBoost) in demand-driven acquisition (DDA) prediction: A machine-learning approach

Walker, Kevin W.; Jiang, Zhehan

doi:10.1016/j.acalib.2019.02.013

“…Zarandi et al [4] used the AdaBoost with Support Vector Regressor in modeling minimum miscibility pressure of pure/impure CO2-crude oil systems and concluded that the model gives the most acceptable and accurate result with a very satisfactory error distribution. AdaBoost was also implemented by Walker & Jiang [23] in the demand-driven acquisition of library materials and compared it to the logistic regression model. The authors concluded that AdaBoost performs better with an accuracy of 82%.…”

Section: Related Workmentioning

confidence: 99%

“…To perform the classification of risk level in the data, the following steps were presented in the pseudocode. The algorithm searches for the candidate support vectors represented by S and assumes that SV occupies a space where the parameters of the linear features of the hyper-plane are stored [23].…”

Section: Support Vector Machinementioning

confidence: 99%

Classification of Disaster Risks in the Philippines using Adaptive Boosting Algorithm with Decision Trees and Support Vector Machine as Based Estimators

Acula

¹

2021

J. Mod. Sim. Mater.

4

0

View full text Add to dashboard Cite

This paper employed the intelligent approach based on machine learning categorized as base and ensemble methods in classifying the disaster risk in the Philippines. It focused on the Decision Trees, Support Vector Machine, Adaptive Boosting Algorithm with Decision Trees, and Support Vector Machine as base estimators. The research used the Exponential Regression for missing value imputation and converted the number of casualties, damaged houses, and properties into five (5) risk levels using Quantile Method. The 10-fold cross-validation was used to validate the proposed algorithms. The experiment shows that Decision Trees and Adaptive Decision Trees are the most suitable models for the disaster data with the score of more than 90%, more than 75%, more than 75% in all the classification metrics (accuracy, precision, recall f1-score) when applied to classification risk levels of casualties, damaged houses and damaged properties respectively.

show abstract

“…It is one of the most significant developments in Machine Learning [50,51]. AdaBoost [52] was the first, widely used implementation of boosting and is still favoured for its accuracy, ease of deployment and fast training time [53,54,55]. It uses shallow decision trees as the weak classifiers.…”

Section: Multi-class Adaboostmentioning

confidence: 99%

Ada-WHIPS: Explaining AdaBoost Classification with Applications in the Health Sciences

Hatwell

¹

,

Gaber

²

,

Azad

³

2020

Preprint

1

0

View full text Add to dashboard Cite

Background Computer Aided Diagnostics (CAD) can support medical practitioners to make critical decisions about their patients' disease conditions. Practitioners require access to the chain of reasoning behind CAD to build trust in the CAD advice and to supplement their own expertise. Yet, CAD systems might be based on black box machine learning (ML) models and high dimensional data sources (electronic health records, MRI scans, cardiotocograms, etc). These foundations make interpretation and explanation of the CAD advice very challenging. This challenge is recognised throughout the machine learning research community. eXplainable Artificial Intelligence (XAI) is emerging as one of the most important research areas of recent years, because it addresses the interpretability and trust concerns of medical practitioners and other critical decision makers. Method In this work, we focus on AdaBoost, a black box model that has been widely adopted in the CAD literature. We address the challenge -- to explain AdaBoost classification -- with a novel algorithm that extracts simple, logical rules from AdaBoost models. Our algorithm, \textit{Adaptive-Weighted High Importance Path Snippets} (Ada-WHIPS), makes use of AdaBoost's adaptive classifier weights; using a novel formulation, Ada-WHIPS uniquely redistributes the weights among individual decision nodes at the internals of the AdaBoost model. Then, a simple heuristic search of the weighted nodes finds a single rule that dominated the model's decision. We compare the explanations generated by our novel approach with the state of the art in an experimental study. We evaluate the derived explanations with simple statistical tests of well-known quality measures, precision and coverage, and a novel measure \textit{stability} that is better suited to the XAI setting. Results In this paper, our experimental results demonstrate the benefits of using our novel algorithm for explaining AdaBoost classification. The simple rule-based explanations have better generalisation (mean coverage 15\%-68\%) while remaining competitive for specificity (mean precision 80\%-99\%). A very small trade-off in specificity is shown to guard against over-fitting. Conclusions This research demonstrates that interpretable, classification rule-based explanations can be generated for computer aided diagnostic tools based on AdaBoost, and that a tightly coupled, AdaBoost-specific approach can outperform model-agnostic methods.

show abstract

“…It is one of the most significant developments in Machine Learning [46,47]. AdaBoost [48] was the first, widely used implementation of boosting and is still favoured for its accuracy, ease of deployment and fast training time [49,50,51]. It uses shallow decision trees as the weak classifiers.…”

Section: Multi-class Adaboostmentioning

confidence: 99%

Ada-WHIPS: Explaining AdaBoost Classification with Applications in the Health Sciences

Hatwell

¹

,

Gaber

²

,

Azad

³

2019

Preprint

0

View full text Add to dashboard Cite

Background Computer Aided Diagnostics (CAD) can support medical practitioners to make critical decisions about their patients' disease conditions. Practitioners require access to the chain of reasoning behind CAD to build trust in the CAD advice and to supplement their own expertise. Yet, CAD systems might be based on black box machine learning (ML) models and high dimensional data sources (electronic health records, MRI scans, cardiotocograms, etc). These foundations make interpretation and explanation of the CAD advice very challenging. This challenge is recognised throughout the machine learning research community. eXplainable Artificial Intelligence (XAI) is emerging as one of the most important research areas of recent years, because it addresses the interpretability and trust concerns of medical practitioners and other critical decision makers. Method In this work, we focus on AdaBoost, a black box model that has been widely adopted in the CAD literature. We address the challenge -- to explain AdaBoost classification -- with a novel algorithm that extracts simple, logical rules from AdaBoost models. Our algorithm, \textit{Adaptive-Weighted High Importance Path Snippets} (Ada-WHIPS), makes use of AdaBoost's adaptive classifier weights; using a novel formulation, Ada-WHIPS uniquely redistributes the weights among individual decision nodes at the internals of the AdaBoost model. Then, a simple heuristic search of the weighted nodes finds a single rule that dominated the model's decision. We compare the explanations generated by our novel approach with the state of the art in an experimental study. We evaluate the derived explanations with simple statistical tests of well-known quality measures, precision and coverage, and a novel measure \textit{stability} that is better suited to the XAI setting. Results In this paper, our experimental results demonstrate the benefits of using our novel algorithm for explaining AdaBoost classification. The simple rule-based explanations have better generalisation (mean coverage 15\%-68\%) while remaining competitive for specificity (mean precision 80\%-99\%). A very small trade-off in specificity is shown to guard against over-fitting. Conclusions This research demonstrates that interpretable, classification rule-based explanations can be generated for computer aided diagnostic tools based on AdaBoost, and that a tightly coupled, AdaBoost-specific approach can outperform model-agnostic methods.

show abstract

Application of adaptive boosting (AdaBoost) in demand-driven acquisition (DDA) prediction: A machine-learning approach

Cited by 48 publications

References 33 publications

Classification of Disaster Risks in the Philippines using Adaptive Boosting Algorithm with Decision Trees and Support Vector Machine as Based Estimators

Classification of Disaster Risks in the Philippines using Adaptive Boosting Algorithm with Decision Trees and Support Vector Machine as Based Estimators

Ada-WHIPS: Explaining AdaBoost Classification with Applications in the Health Sciences

Ada-WHIPS: Explaining AdaBoost Classification with Applications in the Health Sciences

Contact Info

Product

Resources

About