Evaluating Models' Local Decision Boundaries via Contrast Sets

Gardner, Matt; Artzi, Yoav; Basmova, Victoria; Berant, Jonathan; Bogin, Ben; Chen, Sihao; Dasigi, Pradeep; Dua, Dheeru; Elazar, Yanai; Gottumukkala, Ananth; Gupta, Neelesh; Hajishirzi, Hanna; Ilharco, Gabriel; Khashabi, Daniel; Lin, Kevin; Liu, Jiangming; Liu, Nelson F.; Mulcaire, Phoebe; Ning, Qiang; Singh, Sameer; Smith, Noah A.; Subramanian, Sanjay; Tsarfaty, Reut; Wallace, Eric; Zhang, Ally; Zhou, Ben

doi:10.48550/arxiv.2004.02709

Cited by 37 publications

(19 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a metric, the attack success rate has been utilized in numerous research studies to determine the effectiveness of adversarial attacks on NLP models [12], [15], [39], [115], [116]. For instance, Alzantot et al [39] measured the effectiveness of their genetic algorithm-based adversarial attacks using the attack success rate as a metric which eventually indicates NLP model robustness to adversarial examples.…”

Section: A Attack Success Ratementioning

confidence: 99%

Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions

Omar¹,

Choi²,

Nyang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent natural language processing (NLP) techniques have accomplished high performance on benchmark datasets, primarily due to the significant improvement in the performance of deep learning. The advances in the research community have led to great enhancements in state-of-the-art production systems for NLP tasks, such as virtual assistants, speech recognition, and sentiment analysis. However, such NLP systems still often fail when tested with adversarial attacks. The initial lack of robustness exposed troubling gaps in current models' language understanding capabilities, creating problems when NLP systems are deployed in real life. In this paper, we present a structured overview of NLP robustness research by summarizing the literature in a systemic way across various dimensions. We then take a deep-dive into the various dimensions of robustness, across techniques, metrics, embeddings, and benchmarks. Finally, we argue that robustness should be multi-dimensional, provide insights into current research, identify gaps in the literature to suggest directions worth pursuing to address these gaps.

show abstract

Section: A Attack Success Ratementioning

confidence: 99%

Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions

Omar¹,

Choi²,

Nyang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Firstly, the typical way of structural analysis is to design probe classifiers to analyze model characteristics, such as syntactic structural features (Elazar et al, 2021) and semantic features (Wu et al, 2021). Secondly, the main idea of behavioral studies is that design experiments that allow researchers to make inferences about computed representations based on the model's behavior, such as proposing various challenge sets that aim to cover specific, diverse phenomena, like systematicity exhaustivity (Gardner et al, 2020;Ravichander et al, 2021). Thirdly, for interactive visualization, neuron activation (Durrani et al, 2020), attention mechanisms (Hao et al, 2020) and saliency measures (Janizek et al, 2021) are three main standard visualization methods.…”

Section: Related Workmentioning

confidence: 99%

Ask to Understand: Question Generation for Multi-hop Question Answering

Li¹,

Ren²,

Gao³

et al. 2022

Preprint

View full text Add to dashboard Cite

Multi-hop Question Answering (QA) requires the machine to answer complex questions by finding scattering clues and reasoning from multiple documents. Graph Network (GN) and Question Decomposition (QD) are two common approaches at present. The former uses the "black-box" reasoning process to capture the potential relationship between entities and sentences, thus achieving good performance. At the same time, the latter provides a clear reasoning logical route by decomposing multi-hop questions into simple singlehop sub-questions. In this paper, we propose a novel method to complete multi-hop QA from the perspective of Question Generation (QG). Specifically, we carefully design an endto-end QG module on the basis of a classical QA module, which could help the model understand the context by asking inherently logical sub-questions, thus inheriting interpretability from the QD-based method and showing superior performance. Experiments on the HotpotQA dataset demonstrate that the effectiveness of our proposed QG module, human evaluation further clarifies its interpretability quantitatively, and thorough analysis shows that the QG module could generate better subquestions than QD methods in terms of fluency, consistency, and diversity."Educated mind first sign is good at asking questions."-Plekhanov G.V.

show abstract

“…Annotation artifacts (Gururangan et al, 2018) are gaps present in a dataset that can lead to misleading interpretations of a model's performance on that dataset. To mitigate this, we evaluate F on contrast sets (Gardner et al, 2020), which are (mostly) labelchanging small perturbations on instances to understand the true local boundary of the dataset. Essentially, they help us understand if F has learnt any dataset-specific shortcuts.…”

Section: Contrast Set Testsmentioning

confidence: 99%

“…For contrast set tests, we use an OOD contrast set for sentiment analysis released by the authors of the original paper (Gardner et al, 2020), which are created for the Movies dataset. Furthermore, for functional tests, we use an OOD test suite (flight reviews) from the CheckList (Ribeiro et al, 2020) which contains both template-based instances to test linguistic capabilities, as well as real-world data (tweets).…”

Section: Tasks and Datasetsmentioning

confidence: 99%

ER-Test: Evaluating Explanation Regularization Methods for Language Models

Joshi¹,

Chan²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Neural language models' (NLMs') reasoning processes are notoriously hard to explain. Recently, there has been much progress in automatically generating machine rationales of NLM behavior, but less in utilizing the rationales to improve NLM behavior. For the latter, explanation regularization (ER) aims to improve NLM generalization by pushing the machine rationales to align with human rationales. Whereas prior works primarily evaluate such ER models via in-distribution (ID) generalization, ER's impact on out-of-distribution (OOD) is largely underexplored. Plus, little is understood about how ER model performance is affected by the choice of ER criteria or by the number/choice of training instances with human rationales. In light of this, we propose ER-TEST, a protocol for evaluating ER models' OOD generalization along three dimensions: (1) unseen datasets, (2) contrast set tests, and (3) functional tests. Using ER-TEST, we study three key questions: (A) Which ER criteria are most effective for the given OOD setting? (B) How is ER affected by the number/choice of training instances with human rationales? (C) Is ER effective with distantly supervised human rationales? ER-TEST enables comprehensive analysis of these questions by considering a diverse range of tasks and datasets. Through ER-TEST, we show that ER has little impact on ID performance, but can yield large gains on OOD performance w.r.t. ( 1)-(3). Also, we find that the best ER criterion is task-dependent, while ER can improve OOD performance even with limited and distantly-supervised human rationales.

show abstract

Evaluating Models' Local Decision Boundaries via Contrast Sets

Cited by 37 publications

References 55 publications

Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions

Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions

Ask to Understand: Question Generation for Multi-hop Question Answering

ER-Test: Evaluating Explanation Regularization Methods for Language Models

Contact Info

Product

Resources

About