Investigating the impact of calibration on the quality of explanations

Löfström, Helena; Löfström, Tuwe; Johansson, Ulf; Sönströd, Cecilia

doi:10.1007/s10472-023-09837-2

Cited by 3 publications

(6 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The major findings of the study argue that local explanations significantly improve user trust and AI accuracy even more than confidence score. The main study by Löfström et al [73] investigates the impacts of the calibration in the explanation quality by utilizing misleading explainers such as LIME. The study indicates that adding a layer of calibration will significantly enhance the accuracy and fidelity of explanations.…”

Section: Discussionmentioning

confidence: 99%

Explainable AI Evaluation: A Top-Down Approach for Selecting Optimal Explanations for Black Box Models

Mirzaei,

Mao,

Al-Nima

et al. 2023

Information

View full text Add to dashboard Cite

Explainable Artificial Intelligence (XAI) evaluation has grown significantly due to its extensive adoption, and the catastrophic consequence of misinterpreting sensitive data, especially in the medical field. However, the multidisciplinary nature of XAI research resulted in diverse scholars possessing significant challenges in designing proper evaluation methods. This paper proposes a novel framework of a three-layered top-down approach on how to arrive at an optimal explainer, accenting the persistent need for consensus in XAI evaluation. This paper also investigates a critical comparative evaluation of explanations in both model agnostic and specific explainers including LIME, SHAP, Anchors, and TabNet, aiming to enhance the adaptability of XAI in a tabular domain. The results demonstrate that TabNet achieved the highest classification recall followed by TabPFN, and XGBoost. Additionally, this paper develops an optimal approach by introducing a novel measure of relative performance loss with emphasis on faithfulness and fidelity of global explanations by quantifying the extent to which a model’s capabilities diminish when eliminating topmost features. This addresses a conspicuous gap in the lack of consensus among researchers regarding how global feature importance impacts classification loss, thereby undermining the trust and correctness of such applications. Finally, a practical use case on medical tabular data is provided to concretely illustrate the findings.

show abstract

Section: Discussionmentioning

confidence: 99%

Explainable AI Evaluation: A Top-Down Approach for Selecting Optimal Explanations for Black Box Models

Mirzaei,

Mao,

Al-Nima

et al. 2023

Information

View full text Add to dashboard Cite

show abstract

“…The implementation of both the regression and the probabilistic regression solutions is expanding the calibrated-explanations Python package [23] and relies on the ConformalPredictiveSystem from the crepes package [37]. By default, ConformalPredictiveSystem is used without normalization but DifficultyEstimator provided by crepes.extras is fully supported by calibrated-explanations, with normalization options corresponding to the list given at the end of Section 3.1.…”

Section: Methodsmentioning

confidence: 99%

“…Below is an introduction to CEC [23], which provides the foundation to this paper's contribution 1 . In the following descriptions, a factual explanation is composed of a calibrated prediction from the underlying model accompanied by an uncertainty interval and a collection of factual feature rules, each composed of a feature weight with an uncertainty interval and a factual condition, covering that feature's instance value.…”

Section: Calibrated Explanations For Classification (Cec)mentioning

confidence: 99%

“…Hence, the discretizer is typically binary for normal use of CE. For that reason, two binary discretizers were introduced in [23] to complement the discretizers existing in LIME:…”

Section: Declarationsmentioning

confidence: 99%

“…Being able to quantify the uncertainty of feature weights can improve the quality and usefulness of explanations in XAI. Recently, a local explanation method, Calibrated Explanations (CE), utilizing the intervals provided by VA to estimate feature uncertainty was introduced for classification [23].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Calibrated Explanations for Regression

Löfström,

Johansson

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Artificial Intelligence (AI) is often an integral part of modern decision support systems (DSS). The best-performing predictive models used in AI-based DSS lack transparency. Explainable Artificial Intelligence (XAI) aims to create AI systems that can explain their rationale to human users. Local explanations in XAI can provide information about the causes of individual predictions in terms of feature importance. However, a critical drawback of existing local explanation methods is their inability to quantify the uncertainty associated with a feature's importance. This paper introduces an extension of a feature importance explanation method, Calibrated Explanations (CE), previously only supporting classification, with support for standard regression and probabilistic regression, i.e., the probability that the target is above an arbitrary threshold. The extension for regression keeps all the benefits of CE, such as calibration of the prediction from the underlying model with confidence intervals, uncertainty quantification of feature importance, and allows both factual and counterfactual explanations. CE for standard regression provides fast, reliable, stable, and robust explanations. CE for probabilistic regression provides an entirely new way of creating probabilistic explanations from any ordinary regression model, allowing dynamic selection of thresholds. The performance of CE for probabilistic regression regarding stability and speed is comparable to LIME. The method is model agnostic with easily understood conditional rules. An implementation in Python is freely available on GitHub and for installation using both pip and conda, making the results in this paper easily replicable.

show abstract

Identifying Frailty in Older Adults Receiving Home Care Assessment Using Machine Learning: Longitudinal Observational Study on the Role of Classifier, Feature Selection, and Sample Size

Pan,

Luo,

Cheung

et al. 2024

JMIR AI

View full text Add to dashboard Cite

Background Machine learning techniques are starting to be used in various health care data sets to identify frail persons who may benefit from interventions. However, evidence about the performance of machine learning techniques compared to conventional regression is mixed. It is also unclear what methodological and database factors are associated with performance. Objective This study aimed to compare the mortality prediction accuracy of various machine learning classifiers for identifying frail older adults in different scenarios. Methods We used deidentified data collected from older adults (65 years of age and older) assessed with interRAI-Home Care instrument in New Zealand between January 1, 2012, and December 31, 2016. A total of 138 interRAI assessment items were used to predict 6-month and 12-month mortality, using 3 machine learning classifiers (random forest [RF], extreme gradient boosting [XGBoost], and multilayer perceptron [MLP]) and regularized logistic regression. We conducted a simulation study comparing the performance of machine learning models with logistic regression and interRAI Home Care Frailty Scale and examined the effects of sample sizes, the number of features, and train-test split ratios. Results A total of 95,042 older adults (median age 82.66 years, IQR 77.92-88.76; n=37,462, 39.42% male) receiving home care were analyzed. The average area under the curve (AUC) and sensitivities of 6-month mortality prediction showed that machine learning classifiers did not outperform regularized logistic regressions. In terms of AUC, regularized logistic regression had better performance than XGBoost, MLP, and RF when the number of features was ≤80 and the sample size ≤16,000; MLP outperformed regularized logistic regression in terms of sensitivities when the number of features was ≥40 and the sample size ≥4000. Conversely, RF and XGBoost demonstrated higher specificities than regularized logistic regression in all scenarios. Conclusions The study revealed that machine learning models exhibited significant variation in prediction performance when evaluated using different metrics. Regularized logistic regression was an effective model for identifying frail older adults receiving home care, as indicated by the AUC, particularly when the number of features and sample sizes were not excessively large. Conversely, MLP displayed superior sensitivity, while RF exhibited superior specificity when the number of features and sample sizes were large.

show abstract

Investigating the impact of calibration on the quality of explanations

Cited by 3 publications

References 24 publications

Explainable AI Evaluation: A Top-Down Approach for Selecting Optimal Explanations for Black Box Models

Explainable AI Evaluation: A Top-Down Approach for Selecting Optimal Explanations for Black Box Models

Calibrated Explanations for Regression

Identifying Frailty in Older Adults Receiving Home Care Assessment Using Machine Learning: Longitudinal Observational Study on the Role of Classifier, Feature Selection, and Sample Size

Contact Info

Product

Resources

About