Developing a Fidelity Evaluation Approach for Interpretable Machine Learning

Velmurugan, Mythreyi; Ouyang, Chun; Moreira, Catarina; Sindhgatta, Renuka

doi:10.48550/arxiv.2106.08492

Cited by 3 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fidelity of interpretability method (AI experts)—uses two metrics ( Velmurugan et al, 2021 ), recall and precision , where the term True Features (TF) represents the relevant features as extracted directly from the model and Explanation Features (EF) represents the features characterised as most relevant…”

Section: Explainability and Interpretabilitymentioning

confidence: 99%

Explainability of deep learning models in medical video analysis: a survey

Kolárik

Sarnovský

Paralič

et al. 2023

PeerJ Computer Science

View full text Add to dashboard Cite

Deep learning methods have proven to be effective for multiple diagnostic tasks in medicine and have been performing significantly better in comparison to other traditional machine learning methods. However, the black-box nature of deep neural networks has restricted their use in real-world applications, especially in healthcare. Therefore, explainability of the machine learning models, which focuses on providing of the comprehensible explanations of model outputs, may affect the possibility of adoption of such models in clinical use. There are various studies reviewing approaches to explainability in multiple domains. This article provides a review of the current approaches and applications of explainable deep learning for a specific area of medical data analysis—medical video processing tasks. The article introduces the field of explainable AI and summarizes the most important requirements for explainability in medical applications. Subsequently, we provide an overview of existing methods, evaluation metrics and focus more on those that can be applied to analytical tasks involving the processing of video data in the medical domain. Finally we identify some of the open research issues in the analysed area.

show abstract

Section: Explainability and Interpretabilitymentioning

confidence: 99%

Explainability of deep learning models in medical video analysis: a survey

Kolárik

Sarnovský

Paralič

et al. 2023

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…In this paper, we measure faithfulness to the model. Earlier work has looked at global measures of this type [48] and measures that are specialized to neural networks [32], feature importance [4,9,43,45], rule-based explanations [24], surrogate explanation [35], or highlighted text [10,46,49].…”

Section: Related Workmentioning

confidence: 99%

Framework for Evaluating Faithfulness of Local Explanations

Dasgupta¹,

Frost²,

Moshkovitz³

2022

Preprint

View full text Add to dashboard Cite

We study the faithfulness of an explanation system to the underlying prediction model. We show that this can be captured by two properties, consistency and sufficiency, and introduce quantitative measures of the extent to which these hold. Interestingly, these measures depend on the test-time data distribution. For a variety of existing explanation systems, such as anchors, we analytically study these quantities. We also provide estimators and sample complexity bounds for empirically determining the faithfulness of black-box explanation systems. Finally, we experimentally validate the new properties and estimators.

show abstract

“…Surveys about the evaluation of XAI methods were published by [13,53]. Defining appropriate and generally applicable metrics to evaluate the quality of explanations [3,31,[46][47][48], the field is actively researched and hasn't converged towards a set of standard metrics.…”

Section: Explainable Ai Evaluation Frameworkmentioning

confidence: 99%

RESHAPE: Explaining Accounting Anomalies in Financial Statement Audits by enhancing SHapley Additive exPlanations

Müller¹,

Schreyer²,

Sattarov³

et al. 2022

Preprint

View full text Add to dashboard Cite

Detecting accounting anomalies is a recurrent challenge in financial statement audits. Recently, novel methods derived from Deep-Learning (DL) have been proposed to audit the large volumes of a statement's underlying accounting records. However, due to their vast number of parameters, such models exhibit the drawback of being inherently opaque. At the same time, the concealing of a model's inner workings often hinders its real-world application. This observation holds particularly true in financial audits since auditors must reasonably explain and justify their audit decisions. Nowadays, various Explainable AI (XAI) techniques have been proposed to address this challenge, e.g., SHapley Additive exPlanations (SHAP). However, in unsupervised DL as often applied in financial audits, these methods explain the model output at the level of encoded variables. As a result, the explanations of Autoencoder Neural Networks (AENNs) are often hard to comprehend by human auditors. To mitigate this drawback, we propose Reconstruction Error SHapley Additive exPlanations Extension (RESHAPE), which explains the model output on an aggregated attribute-level. In addition, we introduce an evaluation framework to compare the versatility of XAI methods in auditing. Our experimental results show empirical evidence that RESHAPE results in versatile explanations compared to state-of-the-art baselines. We envision such attribute-level explanations as a necessary next step in the adoption of unsupervised DL techniques in financial auditing. CCS CONCEPTS• Computing methodologies → Machine learning; • Information systems → Enterprise resource planning.

show abstract

Developing a Fidelity Evaluation Approach for Interpretable Machine Learning

Cited by 3 publications

References 22 publications

Explainability of deep learning models in medical video analysis: a survey

Explainability of deep learning models in medical video analysis: a survey

Framework for Evaluating Faithfulness of Local Explanations

RESHAPE: Explaining Accounting Anomalies in Financial Statement Audits by enhancing SHapley Additive exPlanations

Contact Info

Product

Resources

About