Towards Trustworthy Predictions from Deep Neural Networks with Fast Adversarial Calibration

Abstract

To facilitate a wide-spread acceptance of AI systems guiding decision making in real-world applications, trustworthiness of deployed models is key. That is, it is crucial for predictive models to be uncertainty-aware and yield well-calibrated (and thus trustworthy) predictions for both in-domain samples as well as under domain shift. Recent efforts to account for predictive uncertainty include post-processing steps for trained neural networks, Bayesian neural networks as well as alternative non-Bayesian approaches such as ensemble approaches and evidential deep learning. Here, we propose an efficient yet general modelling approach for obtaining well-calibrated, trustworthy probabilities for samples obtained after a domain shift. We introduce a new training strategy combining an entropy-encouraging loss term with an adversarial calibration loss term and demonstrate that this results in well-calibrated and technically trustworthy predictions for a wide range of domain drifts. We comprehensively evaluate previously proposed approaches on different data modalities, a large range of data sets including sequence data, network architectures and perturbation strategies. We observe that our modelling approach substantially outperforms existing state-of-the-art approaches, yielding well-calibrated predictions under domain drift.

Improved post-hoc probability calibration for out-of-domain MRI segmentation

“…While with the temperature map T i ∈ R C×M ×N , the calibrated probability ŷc i can be obtained by re-scaling the logits using T i , i.e. ŷc i = σ(z i /T i ) 7 . Method overview: We aim to obtain a temperature-scaling-based calibration network g φ (•) that is suitable for out-of-domain (OOD) testing images.…”

“…Unfortunately, most existing probability calibration methods cannot be directly applied to medical image segmentation due to the following reasons: First, the majority of existing methods are designed for image classification, which yield a single class probability per image [4,5,6,7,8]. Secondly, most previous methods assume training and testing images are from a same domain.…”

A Novel Gradient Accumulation Method for Calibration of Named Entity Recognition Models

“…Mitrose et al [22] compared in and out of domain examples using the cosine distance and used it as a regularization loss. Taking inspiration from the ECE, Tomani et al [34] suggested an adversarial loss term. A different training procedure is suggested by Noh et al [26] where examples are processed several times by the model with a different a dropout mask to produce different gradients which are then averaged.…”

Propagating variational model uncertainty for bioacoustic call label smoothing

“…Unfortunately, during this near-decade of DL advancement, a fixation by the DL community toward deeper and more complicated architectures, as well as on traditional prediction performance evaluation measures, has led to an insidious DL model behavior manifesting overconfident predictions, 30 i.e., predictions made at a probability nearing 1, regardless of whether they are correct or not. Downstream software modules or policy makers making catastrophic decisions due to these overconfidently predicted misclassifications can foster deep mistrust in DL, 30 , 31 , 32 something that has also been noted with respect to bioacoustics. 3 However, early prediction calibration fixes 30 are based on learning a transformation of the model outputs that requires the existence of a validation set of labels, something that cannot be safely assumed in general.…”