Explaining Prediction Uncertainty of Pre-trained Language Models by Detecting Uncertain Words in Inputs

Abstract

Estimating the predictive uncertainty of pretrained language models is important for increasing their trustworthiness in NLP. Although many previous works focus on quantifying prediction uncertainty, there is little work on explaining the uncertainty. This paper pushes a step further on explaining uncertain predictions of post-calibrated pre-trained language models. We adapt two perturbationbased post-hoc interpretation methods, Leaveone-out and Sampling Shapley, to identify words in inputs that cause the uncertainty in predictions. We test the proposed methods on BERT and RoBERTa with three tasks: sentiment classification, natural language inference, and paraphrase identification, in both indomain and out-of-domain settings. Experiments show that both methods consistently capture words in inputs that cause prediction uncertainty.

Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcity

“…Shen et al (2020) attempt to transfer the idea of prior networks (Malinin and Gales, 2018;Joo et al, 2020) onto recurrent neural networks. Another line of works investigates uncertainty properties themselves; For instance, Chen and Ji (2022) try to explain uncertainty estimates for BERT and RoBERTa. Another example is given by Xiao and Wang (2021), who use predictive uncertainty to explain hallucination in Language Generation.…”