Evaluating Explanations: How much do explanations from the teacher aid students?

Pruthi, Danish; Bansal, Rachit; Dhingra, Bhuwan; Soares, Livio Baldini; Collins, Michael J.; Neubig, Graham; Cohen, William W.

doi:10.48550/arxiv.2012.00893

Cited by 17 publications

(37 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To improve the model's learning, explanations can be used in a diverse range of ways, including as extra supervision or regularization (Pruthi et al, 2020;Hase et al, 2020;Narang et al, 2020;Andreas et al, 2017), pruned inputs (Jain et al, 2020;Bastings et al, 2019;Lei et al, 2016), additional inputs (Hase and Bansal, 2021;Co-Reyes et al, 2018), and intermediate variables Zhou et al, 2020;. The most similar work to ours is Pruthi et al (2020), which proposed using extractive text explanations to regularize a PLM's self-attention mechanism and demonstrated considerable performance gains. Still, methods for learning from explanations have largely focused on domains like text and images, as opposed to graphs.…”

Section: Learning From Model Explanationsmentioning

confidence: 84%

“…However, it is not necessarily obvious how these KG explanations should be used, especially for improving the model. Recently, Pruthi et al (2020) proposed using text saliency explanations to regularize a PLM's self-attention mechanism, then evaluating explanations by their ability to improve the PLM's performance. Inspired by this idea, we view KG explanations as rich signals for teaching KG-augmented models how to filter out task-irrelevant KG information.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning

Chan

Xu²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

Augmenting pre-trained language models with knowledge graphs (KGs) has achieved success on various commonsense reasoning tasks. Although some works have attempted to explain the behavior of such KG-augmented models by indicating which KG inputs are salient (i.e., important for the model's prediction), it is not always clear how these explanations should be used to make the model better. In this paper, we explore whether KG explanations can be used as supervision for teaching these KG-augmented models how to filter out unhelpful KG information. To this end, we propose SALKG, a simple framework for learning from KG explanations of both coarse (Is the KG salient?) and fine (Which parts of the KG are salient?) granularity. Given the explanations generated from a task's training set, SALKG trains KG-augmented models to solve the task by focusing on KG information highlighted by the explanations as salient. Across two popular commonsense QA benchmarks and three KG-augmented models, we find that SALKG's training process can consistently improve model performance. 1 * Equal contribution. † Work done while TG interned remotely at USC. 1 Code will be released at github.com/INK-USC/SalKG.

show abstract

Section: Learning From Model Explanationsmentioning

confidence: 84%

Section: Introductionmentioning

confidence: 99%

SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning

Chan

Xu²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A variety of automated metrics to measure explanation quality have been proposed in the past. However, many of them can be easily gamed (Hooker et al 2019;Treviso and Martins 2020b;Hase et al 2020a) (see (Pruthi et al 2020) for a detailed discussion on this point). A popular way to evaluate explanations is to compare the produced explanations with expert-collected rationales (Mullenbach et al 2018;DeYoung et al 2020).…”

Section: Related Workmentioning

confidence: 99%

“…Chandrasekaran et al (2018) present model explanations during the testing phase, whereas do not include explanations at test time, as explanations could "leak" model output (seePruthi et al (2020);Jacovi and Goldberg (2020). ).…”

mentioning

confidence: 99%

Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations

Arora¹,

Pruthi²,

Sadeh³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In attempts to "explain" predictions of machine learning models, researchers have proposed hundreds of techniques for attributing predictions to features that are deemed important. While these attributions are often claimed to hold the potential to improve human "understanding" of the models, surprisingly little work explicitly evaluates progress towards this aspiration. In this paper, we conduct a crowdsourcing study, where participants interact with deception detection models that have been trained to distinguish between genuine and fake hotel reviews. They are challenged both to simulate the model on fresh reviews, and to edit reviews with the goal of lowering the probability of the originally predicted class. Successful manipulations would lead to an adversarial example. During the training (but not the test) phase, input spans are highlighted to communicate salience. Through our evaluation, we observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control. For the BERT-based classifier, popular local explanations do not improve their ability to reduce the model confidence over the no-explanation case. Remarkably, when the explanation for the BERT model is given by the (global) attributions of a linear model trained to imitate the BERT model, people can effectively manipulate the model. 1 * denotes equal contribution.

show abstract

“…Explainers are compared with count-based metrics (Poerner et al, 2018;De Cao et al, 2020;Tsang et al, 2020;Nguyen and Martínez, 2020;Bodria et al, 2021;Ding and Koehn, 2021;Yin et al, 2021;Hase et al, 2021;Kokhlikyan et al, 2021;Zafar et al, 2021;Sinha et al, 2021) and against human judgement (Nguyen, 2018;Lertvittayakumjorn and Toni, 2019;Hase and Bansal, 2020;Prasad et al, 2020). Feature attribution scores have also been incorporated into model training (Ross et al, 2017;Liu and Avci, 2019;Erion et al, 2021;Pruthi et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

Thermostat: A Large Collection of NLP Model Explanations and Analysis Tools

Feldhus¹,

Schwarzenberg²,

Möller³

2021

Preprint

View full text Add to dashboard Cite

In the language domain, as in other domains, neural explainability takes an ever more important role, with feature attribution methods on the forefront. Many such methods require considerable computational resources and expert knowledge about implementation details and parameter choices. To facilitate research, we present THERMOSTAT which consists of a large collection of model explanations and accompanying analysis tools. THER-MOSTAT allows easy access to over 200k explanations for the decisions of prominent stateof-the-art models spanning across different NLP tasks, generated with multiple explainers. The dataset took over 10k GPU hours (> one year) to compile; compute time that the community now saves. The accompanying software tools allow to analyse explanations instance-wise but also accumulatively on corpus level. Users can investigate and compare models, datasets and explainers without the need to orchestrate implementation details.THERMOSTAT is fully open source, democratizes explainability research in the language domain, circumvents redundant computations and increases comparability and replicability.

show abstract

Evaluating Explanations: How much do explanations from the teacher aid students?

Cited by 17 publications

References 0 publications

SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning

SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning

Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations

Thermostat: A Large Collection of NLP Model Explanations and Analysis Tools

Contact Info

Product

Resources

About