Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models

Du, Mengnan; Manjunatha, Varun; Jain, Rajiv; Deshpande, Ruchi; Dernoncourt, Franck; Gu, Jiuxiang; Sun, Tanfeng; Hu, Xia

doi:10.48550/arxiv.2103.06922

Cited by 5 publications

(6 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, in [27], a framework named COSOC was proposed to tackle this shortcut problem by extracting the foreground objects in images to get rid of background-related shortcuts based on a contrastive learning approach. [7] proposed a measurement for quantifying the shortcut degree, with which a shortcut mitigation framework was introduced for natural language understanding (NLU). [47] forces the network to learn the necessary features for all the words in the input to alleviate the shortcut learning problem in supervised Paraphrase Identification (PI).…”

Section: Shortcut Learningmentioning

confidence: 99%

Eye-gaze-guided Vision Transformer for Rectifying Shortcut Learning

Ma¹,

Zhang²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Learning harmful shortcuts such as spurious correlations and biases prevents deep neural networks from learning the meaningful and useful representations, thus jeopardizing the generalizability and interpretability of the learned representation. The situation becomes even more serious in medical imaging, where the clinical data (e.g., MR images with pathology) are limited and scarce while the reliability, generalizability and transparency of the learned model are highly required. To address this problem, we propose to infuse human experts' intelligence and domain knowledge into the training of deep neural networks. The core idea is that we infuse the visual attention information from expert radiologists to proactively guide the deep model to focus on regions with potential pathology and avoid being trapped in learning harmful shortcuts. To do so, we propose a novel eye-gaze-guided vision transformer (EG-ViT) for diagnosis with limited medical image data. We mask the input image patches that are out of the radiologists' interest and add an additional residual connection in the last encoder layer of EG-ViT to maintain the correlations of all patches. The experiments on two public datasets of INbreast and SIIM-ACR demonstrate our EG-ViT model can effectively learn/transfer experts' domain knowledge and achieve much better performance than baselines. Meanwhile, it successfully rectifies the harmful shortcut learning and significantly improves the EG-ViT model's interpretability. In general, EG-ViT takes the advantages of both human expert's prior knowledge and the power of deep neural networks. This work opens new avenues for advancing current artificial intelligence paradigms by infusing human intelligence.

show abstract

Section: Shortcut Learningmentioning

confidence: 99%

Eye-gaze-guided Vision Transformer for Rectifying Shortcut Learning

Ma¹,

Zhang²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Besides, NLI systems relying on superficial syntactic properties (e.g., the lexical overlap heuristic, the subsequence heuristic, the constituent heuristic) may succeed on the majority of examples McCoy et al, 2019;Clark et al, 2019;Utama et al, 2020;Pezeshkpour et al, 2021). On quite a few NLP tasks composed of several components, it has been observed that models fed with partial input can achieve competitive performance compared with those feeding with full input, e.g., leveraging claims without evidence for fact verification (Schuster et al, 2019;Utama et al, 2020;Du et al, 2021b), choosing a plausible story ending for in the narrative cloze test without looking at the story (Cai et al, 2017), question answering based biased positional predictions on the reference document (Jia and Liang, 2017;Kaushik and Lipton, 2018), selecting the appropriate warrant with claims only (without reason) in argument reasoning comprehension (Niven and Kao, 2019;Branco et al, 2021), etc. This paper is the first work to diagnose existing entity typing models whether they have exploited spurious correlations, including the common lexical and the partial input (mention-context) bias, and the task-specific dependency bias.…”

Section: Related Workmentioning

confidence: 99%

Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typing

Xu¹,

Wang²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

The entity typing task aims at predicting one or more words or phrases that describe the type(s) of a specific mention in a sentence. Due to shortcuts from surface patterns to annotated entity labels and biased training, existing entity typing models are subject to the problem of spurious correlations. To comprehensively investigate the faithfulness and reliability of entity typing methods, we first systematically define distinct kinds of model biases that are reflected mainly from spurious correlations. Particularly, we identify six types of existing model biases, including mentioncontext bias, lexical overlapping bias, named entity bias, pronoun bias, dependency bias, and overgeneralization bias. To mitigate these model biases, we then introduce a counterfactual data augmentation method. By augmenting the original training set with their bias-free counterparts, models are forced to fully comprehend the sentences and discover the fundamental cues for entity typing, rather than relying on spurious correlations for shortcuts. Experimental results on the UFET dataset show that our counterfactual data augmentation approach helps improve generalization of different entity typing models with consistently better performance on both in-and out-ofdistribution test sets.

show abstract

“…When biases exist in the dataset, neural networks tend to capture spurious associations and exploit dataset biases as shortcuts to obtain higher evaluation performance instead of truly understanding the language and images, i.e., the models may not learn the intrinsic attributes of better generalization and the ability of reasoning. Recently, several research works 18,19 have shown that intrinsic attributes are usually more difficult to learn than bias attributes, and that in the early training phase of the model, the model first learns shortcut features for fast loss reduction, and only after that the model gradually learns intrinsic attributes for further loss reduction. In short, in the early training stage, the model will first learn to fit the bias-aligned samples, and then gradually fit the bias-conflicting samples.…”

Section: Alleviating Shortcut Learning Behaviormentioning

confidence: 99%

Alleviating shortcut learning behavior of VQA model with context augmentation and adaptive loss adjustment

Zeng

Liu²,

Wang³

2022

Third International Conference on Computer Science and Communication Technology (ICCSCT 2022)

View full text Add to dashboard Cite

Despite the impressive improvements of Visual Question Answer (VQA), it still remains a challenge of how to avoid the suffering of spurious correlations from textual content to answer. Previous researches have shown that due to the existence of language bias in the VQA dataset, VQA models may tend to capture superficial statistical correlation and suffer from the poor generalization capability in the out-of-distribution data. To alleviate the biases caused by language modality, we propose a method of context augmentation and adaptive loss adjustment, which can alleviate shortcut learning behavior of VQA models. Specifically, the existence of language bias is due to the high co-occurrence frequency of categories and the words in “Question”, therefore, we propose to use “Paraphrase Generation” to produce paraphrases with diverse contexts, so as to mitigate such correlation. Secondly, we use adaptive loss adjustment to adjust the importance of samples, that is, reduce the importance of bias-aligned samples and improve the importance of bias-conflicting samples, so as to guide the model to capture the intrinsic attributes that are beneficial to generalization. The experiments have demonstrated the feasibility and validity of our method on a variety of VQA models.

show abstract

Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models

Cited by 5 publications

References 41 publications

Eye-gaze-guided Vision Transformer for Rectifying Shortcut Learning

Eye-gaze-guided Vision Transformer for Rectifying Shortcut Learning

Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typing

Alleviating shortcut learning behavior of VQA model with context augmentation and adaptive loss adjustment

Contact Info

Product

Resources

About