Calibration of Pre-trained Transformers

Abstract

Pre-trained Transformers are now ubiquitous in natural language processing, but despite their high end-task performance, little is known empirically about whether they are calibrated. Specifically, do these models' posterior probabilities provide an accurate empirical measure of how likely the model is to be correct on a given example? We focus on BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019) in this work, and analyze their calibration across three tasks: natural language inference, paraphrase detection, and commonsense reasoning. For each task, we consider in-domain as well as challenging outof-domain settings, where models face more examples they should be uncertain about. We show that: (1) when used out-of-the-box, pretrained models are calibrated in-domain, and compared to baselines, their calibration error out-of-domain can be as much as 3.5× lower;(2) temperature scaling is effective at further reducing calibration error in-domain, and using label smoothing to deliberately increase empirical uncertainty helps calibrate posteriors out-of-domain. 1

How Can We Know What Language Models Know?

“…Previous works that purport to calibrate LMs (Desai and Durrett, 2020;Jagannatha and Yu, 2020) mainly focus on the former use case, using representations learned by LMs to predict target classes (for tasks such as natural language inference or part-of-speech tagging) or identify answer spans (for tasks such as extractive QA). In contrast, we focus on the latter case, calibrating LMs themselves by treating them as natural language generators that predict the next words given a particular input.…”

“…Temperature-based Scaling methods are first proposed on classification tasks (Guo et al, 2017;Desai and Durrett, 2020), where a positive scalar temperature hyperparameter τ is introduced in the final classification layer to make the probability distribution either more peaky or smooth: softmax(z/τ ). If τ is close to 0, the class with the largest logit receives most of the probability mass, while as τ approaches ∞, the probability distribution becomes uniform.…”

“…Calibration Calibration is a well-studied topic in other tasks such as medical diagnosis (Jiang et al, 2012) and image recognition (Guo et al, 2017;Lee et al, 2018). Previous works in NLP have examined calibration in structured prediction problems such as part-of-speech tagging and named entity recognition (Jagannatha and Yu, 2020), natural language understanding tasks such as natural language inference and paraphrase detection (Desai and Durrett, 2020). In contrast, we focus on calibrating LMs themselves by treating them as natural language generators that predict the next words given a particular input.…”

“…Specifically, we fine-tune the LM using softmax-or margin-based objective functions based on multiple candidate answers. For post-hoc calibration, we examined temperaturebased scaling and feature-based decision tree that take prediction probability and input-related features as input and produce calibrated confidence (Jagannatha and Yu, 2020;Desai and Durrett, 2020). We also study the sensitivity of LMs' confidence estimation with respect to language variation by paraphrasing candidate answers and augmenting questions using retrieved context.…”

The Right Tool for the Job: Matching Model and Instance Complexities

“…One way to mitigate this concern is to use calibration, which encourages the confidence level to correspond to the probability that the model is correct (DeGroot and Fienberg, 1983). In this paper we use temperature calibration, which is a simple technique that has been shown to work well in practice (Guo et al, 2017), in particular for BERT fine-tuning (Desai and Durrett, 2020). The method learns a single parameter, denoted temperature or T , and divides each of the logits {z i } by T before applying the softmax function:…”