Evaluating and Aggregating Feature-based Model Explanations

Bhatt, Umang; Weller, Adrian; Moura, J.M.F.

doi:10.24963/ijcai.2020/417

Cited by 91 publications

(72 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As this is rarely available, a typical approach for measuring the faithfulness of the contributions produced by interpretable models is to rely on the proxy notion of the contributions: observing the effect of removing features on the model's prediction. Following previous studies [11], [50], we computed the faithfulness score by removing features one-by-one, measuring the differences between the original predictions and the predictions from the inputs without the removed features, and calculating the correlation between the differences and the contributions of the removed features. Formally, the faithfulness score for set of test samples D te = {(x * , z * )} is calculated as follows:…”

Section: Results On Real Datasetsmentioning

confidence: 99%

Gaussian Process Regression With Interpretable Sample-Wise Feature Weights

Yoshikawa

Iwata

2023

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Gaussian process regression (GPR) is a fundamental model used in machine learning (ML). Due to its accurate prediction with uncertainty and versatility in handling various data structures via kernels, GPR has been successfully used in various applications. However, in GPR, how the features of an input contribute to its prediction cannot be interpreted. Here, we propose GPR with local explanation, which reveals the feature contributions to the prediction of each sample while maintaining the predictive performance of GPR. In the proposed model, both the prediction and explanation for each sample are performed using an easy-to-interpret locally linear model. The weight vector of the locally linear model is assumed to be generated from multivariate Gaussian process priors. The hyperparameters of the proposed models are estimated by maximizing the marginal likelihood. For a new test sample, the proposed model can predict the values of its target variable and weight vector, as well as their uncertainties, in a closed form. Experimental results on various benchmark datasets verify that the proposed model can achieve predictive performance comparable to those of GPR and superior to that of existing interpretable models and can achieve higher interpretability than them, both quantitatively and qualitatively.

show abstract

Section: Results On Real Datasetsmentioning

confidence: 99%

Gaussian Process Regression With Interpretable Sample-Wise Feature Weights

Yoshikawa

Iwata

2023

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…Consequently, approximate methods have been proposed, e.g. aggregation-based methods [ 40 ], Monte Carlo sampling [ 41 ] and approaches for graph-structured data, e.g. language and image data [ 42 ].…”

Section: Explainability Methodsmentioning

confidence: 99%

Artificial intelligence explainability: the technical and ethical dimensions

McDermid

Jia

Porter

et al. 2021

Phil. Trans. R. Soc. A.

View full text Add to dashboard Cite

In recent years, several new technical methods have been developed to make AI-models more transparent and interpretable. These techniques are often referred to collectively as ‘AI explainability’ or ‘XAI’ methods. This paper presents an overview of XAI methods, and links them to stakeholder purposes for seeking an explanation. Because the underlying stakeholder purposes are broadly ethical in nature, we see this analysis as a contribution towards bringing together the technical and ethical dimensions of XAI. We emphasize that use of XAI methods must be linked to explanations of human decisions made during the development life cycle. Situated within that wider accountability framework, our analysis may offer a helpful starting point for designers, safety engineers, service providers and regulators who need to make practical judgements about which XAI methods to employ or to require. This article is part of the theme issue ‘Towards symbiotic autonomous systems’.

show abstract

“…Narayanan et al [92] compared different types of output complexity for how they affected human performance. Bhatt et al [12] designed a complexity metric to quantify "feature importance" explanations.…”

Section: Evaluating Meaningfulnessmentioning

confidence: 99%

“…They then compared the system's scores on the new images. Bhatt et al [12] evaluated the explanation accuracy of "feature importance" explanations by both checking sensitivity, meaning similar inputs have similar feature importance explanations, and faithfulness, meaning the change in the explanations should correlate to the change in inputs.…”

Section: Evaluating Explanation Accuracymentioning

confidence: 99%

Four principles of explainable artificial intelligence

Phillips¹,

Hahn²,

Fontana³

et al. 2021

104

View full text Add to dashboard Cite

We introduce four principles for explainable artificial intelligence (AI) that comprise fundamental properties for explainable AI systems. We propose that explainable AI systems deliver accompanying evidence or reasons for outcomes and processes; provide explanations that are understandable to individual users; provide explanations that correctly reflect the system's process for generating the output; and that a system only operates under conditions for which it was designed and when it reaches sufficient confidence in its output. We have termed these four principles as explanation, meaningful, explanation accuracy, and knowledge limits, respectively. Through significant stakeholder engagement, these four principles were developed to encompass the multidisciplinary nature of explainable AI, including the fields of computer science, engineering, and psychology. Because one-sizefits-all explanations do not exist, different users will require different types of explanations. We present five categories of explanation and summarize theories of explainable AI. We give an overview of the algorithms in the field that cover the major classes of explainable algorithms. As a baseline comparison, we assess how well explanations provided by people follow our four principles. This assessment provides insights to the challenges of designing explainable AI systems.

show abstract

Evaluating and Aggregating Feature-based Model Explanations

Cited by 91 publications

References 1 publication

Gaussian Process Regression With Interpretable Sample-Wise Feature Weights

Gaussian Process Regression With Interpretable Sample-Wise Feature Weights

Artificial intelligence explainability: the technical and ethical dimensions

Four principles of explainable artificial intelligence

Contact Info

Product

Resources

About