Order in the Court: Explainable AI Methods Prone to Disagreement

Neely, Michael J.; F., Schouten, Stefan; Bleeker, Maurits; Lučić, Ana

doi:10.48550/arxiv.2105.03287

Cited by 6 publications

(8 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specialized XAI techniques even exist for adaptive systems such as interactive visualization systems [77], interactive virtual agents [37,83], active learning [27], and human-in-the-loop systems [86] . Furthermore, comparative studies across multiple XAI techniques have shown low mutual agreement [58], which is consistent with internal research findings at AI Research. With these research findings taken together, we expect that different explanation tools will be needed to address each problem domain effectively.…”

Section: The Challengessupporting

confidence: 79%

Seven challenges for harmonizing explainability requirements

Chen,

Storchan

2021

Preprint

View full text Add to dashboard Cite

Regulators have signalled an interest in adopting explainable AI (XAI) techniques to handle the diverse needs for model governance, operational servicing, and compliance in the financial services industry. In this short overview, we review the recent technical literature in XAI and argue that based on our current understanding of the field, the use of XAI techniques in practice necessitate a highly contextualized approach considering the specific needs of stakeholders for particular business applications.

show abstract

Section: The Challengessupporting

confidence: 79%

Seven challenges for harmonizing explainability requirements

Chen,

Storchan

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…50 per class). Subsequent experiments are performed on all validation images or a subset thereof 3 . Hyper-parameters are tuned on validation data as well, as we assume an attacker as well as people looking for explanations have access to the respective data.…”

Section: Datamentioning

confidence: 99%

“…This lack of standard evaluation metrics hinders a comparison between different explanation techniques. Even worse, it was shown that different explanation methods disagree [3,4], which suggests that at least some of them do not capture the real inner workings of a system and that explanations produced by an explainer A cannot serve as a ground truth for a newly developed explainer B.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Constructing adversarial examples to investigate the plausibility of explanations in deep audio and image classifiers

Hoedt¹,

Praher²,

Flexer³

et al. 2022

Neural Comput & Applic

View full text Add to dashboard Cite

Given the rise of deep learning and its inherent black-box nature, the desire to interpret these systems and explain their behaviour became increasingly more prominent. The main idea of so-called explainers is to identify which features of particular samples have the most influence on a classifier’s prediction, and present them as explanations. Evaluating explainers, however, is difficult, due to reasons such as a lack of ground truth. In this work, we construct adversarial examples to check the plausibility of explanations, perturbing input deliberately to change a classifier’s prediction. This allows us to investigate whether explainers are able to detect these perturbed regions as the parts of an input that strongly influence a particular classification. Our results from the audio and image domain suggest that the investigated explainers often fail to identify the input regions most relevant for a prediction; hence, it remains questionable whether explanations are useful or potentially misleading.

show abstract

“…Neely et al (2021) compare DistilBERT based models, we compare BERT-based models. They further constrain their evaluation to 500 instances while we are calculating the values for the entire datasets.…”

mentioning

confidence: 99%

Thermostat: A Large Collection of NLP Model Explanations and Analysis Tools

Feldhus¹,

Schwarzenberg²,

Möller³

2021

Preprint

View full text Add to dashboard Cite

In the language domain, as in other domains, neural explainability takes an ever more important role, with feature attribution methods on the forefront. Many such methods require considerable computational resources and expert knowledge about implementation details and parameter choices. To facilitate research, we present THERMOSTAT which consists of a large collection of model explanations and accompanying analysis tools. THER-MOSTAT allows easy access to over 200k explanations for the decisions of prominent stateof-the-art models spanning across different NLP tasks, generated with multiple explainers. The dataset took over 10k GPU hours (> one year) to compile; compute time that the community now saves. The accompanying software tools allow to analyse explanations instance-wise but also accumulatively on corpus level. Users can investigate and compare models, datasets and explainers without the need to orchestrate implementation details.THERMOSTAT is fully open source, democratizes explainability research in the language domain, circumvents redundant computations and increases comparability and replicability.

show abstract

Order in the Court: Explainable AI Methods Prone to Disagreement

Cited by 6 publications

References 19 publications

Seven challenges for harmonizing explainability requirements

Seven challenges for harmonizing explainability requirements

Constructing adversarial examples to investigate the plausibility of explanations in deep audio and image classifiers

Thermostat: A Large Collection of NLP Model Explanations and Analysis Tools

Contact Info

Product

Resources

About