A Trustworthy View on XAI Method Evaluation

LI, DING; Liu, Yan; Huang, Jun; Wang, Zerui

doi:10.36227/techrxiv.21067438.v1

Cited by 5 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, other methods can also be used to compute metrics related to Faithfulness [16,31]. Robustness measures the stability and consistency of a given XAI method [15,28], while Complexity [15] ensures that explanation would be easily understandable by users. Finally, it can be interesting to measure how explanations coincide with ground truth [28,32].…”

Section: Related Workmentioning

confidence: 99%

Explainability-based Metrics to Help Cyber Operators Find and Correct Misclassified Cyberattacks

Duraz,

Espes,

Francq

et al. 2023

Proceedings of the 2023 on Explainable and Safety Bounded, Fidelitous, Machine Learning for Networking

View full text Add to dashboard Cite

Machine Learning (ML)-based Intrusion Detection Systems (IDS) have shown promising performance. However, in a human-centered context where they are used alongside human operators, there is often a need to understand the reasons of a particular decision. EXplainable AI (XAI) has partially solved this issue, but evaluation of such methods is still difficult and often lacking. This paper revisits two quantitative metrics, Completeness and Correctness, to measure the quality of explanations, i.e., if they properly reflect the actual behaviour of the IDS. Because human operators generally have to handle a huge amount of information in limited time, it is important to ensure that explanations do not miss important causes, and that the important features are indeed causes of an event. However, to be more usable, it is better if explanations are compact. For XAI methods based on feature importance, Completeness shows on some public datasets that explanations tend to point out all important causes only with a high number of features, whereas Correctness seem to be highly correlated with prediction results of the IDS. Finally, besides evaluating the quality of XAI methods, Completeness and Correctness seem to enable identification of IDS failures and can be used to point the operator towards suspicious activity missed or misclassified by the IDS, suggesting manual investigation for correction. CCS CONCEPTS• Security and privacy → Intrusion detection systems; • Humancentered computing → HCI design and evaluation methods;• Computing methodologies → Machine learning.

show abstract

Section: Related Workmentioning

confidence: 99%

Explainability-based Metrics to Help Cyber Operators Find and Correct Misclassified Cyberattacks

Duraz,

Espes,

Francq

et al. 2023

Proceedings of the 2023 on Explainable and Safety Bounded, Fidelitous, Machine Learning for Networking

View full text Add to dashboard Cite

show abstract

“…Meanwhile, features in text content can be evaluated by masking. We pick up Shapley Value [17], SHAP [19], Preddiff [32] and Mean-Centroid Preddiff [33] to perform the feature masking-based explanation.…”

Section: Explanation Consistency Analysismentioning

confidence: 99%

“…In this case, the way of deriving the explanation summary for the Shapley Value, SHAP, and Prediff method is the same as we did in the first case study. For Mean-Centroid Prediff method, the raw explanation for each data instance is the prediction difference in paper [33] as δ x [i] j . This method computes feature contribution value φ j (δ X j ) as the tangent value of the centroid point of clusters formed by the data instances of δ x [i] j .…”

Section: Explanation Consistency Analysismentioning

confidence: 99%

The Analysis and Development of an XAI Process on Feature Contribution Explanation

Huang

Wang

Ding

et al. 2022

2022 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

Explainable Artificial Intelligence (XAI) research focuses on effective explanation techniques to understand and build AI models with trust, reliability, safety, and fairness. Feature importance explanation summarizes feature contributions for end-users to make model decisions. However, XAI methods may produce varied summaries that lead to further analysis to evaluate the consistency across multiple XAI methods on the same model and data set. This paper defines metrics to measure the consistency of feature contribution explanation summaries under feature importance order and saliency map. Driven by these consistency metrics, we develop an XAI process oriented on the XAI criterion of feature importance, which performs a systematical selection of XAI techniques and evaluation of explanation consistency. We demonstrate the process development involving twelve XAI methods on three topics, including a search ranking system, code vulnerability detection and image classification. Our contribution is a practical and systematic process with defined consistency metrics to produce rigorous feature contribution explanations.

show abstract

“…In this case, the way of deriving the explanation summary for the Shapley Value, SHAP, and Prediff method is the same as we did in the first case study. For Mean-Centroid Prediff method, the raw explanation for each data instance is the prediction difference in paper [33] as δ x [i] j . This method computes feature contribution value ϕ j (δ X j ) as the tangent value of the centroid point of clusters formed by the data points of δ x [i] j .…”

Section: Explanation Consistency Analysismentioning

confidence: 99%

The Analysis and Development of an XAI Process on Feature Contribution Explanation

Huang¹,

Wang²,

LI³

et al. 2022

Preprint

View full text Add to dashboard Cite

<p>Explainable Artificial Intelligence (XAI) research focuses on effective explanation techniques to understand and build AI models with trust, reliability, safety, and fairness. Feature importance explanation summarizes feature contributions for end-users to make model decisions. However, XAI methods may produce varied summaries that lead to further analysis to evaluate the consistency across multiple XAI methods on the same model and data set. This paper defines metrics to measure the consistency of feature contribution explanation summaries under feature importance order and saliency map. Driven by these consistency metrics, we develop an XAI process oriented on the XAI criterion of feature importance, which performs a systematical selection of XAI techniques and evaluation of explanation consistency. We demonstrate the process development involving twelve XAI methods on three topics, including a search ranking system, code vulnerability detection and image classification. Our contribution is a practical and systematic process with defined consistency metrics to produce rigorous feature contribution explanations.</p>

show abstract

A Trustworthy View on XAI Method Evaluation

Cited by 5 publications

References 11 publications

Explainability-based Metrics to Help Cyber Operators Find and Correct Misclassified Cyberattacks

Explainability-based Metrics to Help Cyber Operators Find and Correct Misclassified Cyberattacks

The Analysis and Development of an XAI Process on Feature Contribution Explanation

The Analysis and Development of an XAI Process on Feature Contribution Explanation

Contact Info

Product

Resources

About