Imprecise Bayesian Neural Networks

Abstract

Uncertainty quantification and robustness to distribution shifts are important goals in machine learning and artificial intelligence. Although Bayesian neural networks (BNNs) allow for uncertainty in the predictions to be assessed, different sources of uncertainty are indistinguishable. We present imprecise Bayesian neural networks (IBNNs); they generalize and overcome some of the drawbacks of standard BNNs. These latter are trained using a single prior and likelihood distributions, whereas IBNNs are trained using credal prior and likelihood sets. They allow to distinguish between aleatoric and epistemic uncertainties, and to quantify them. In addition, IBNNs are robust in the sense of Bayesian sensitivity analysis, and are more robust than BNNs to distribution shift. They can also be used to compute sets of outcomes that enjoy PAC-like properties. We apply IBNNs to two case studies. One, to model blood glucose and insulin dynamics for artificial pancreas control, and two, for motion prediction in autonomous driving scenarios. We show that IBNNs performs better when compared to an ensemble of BNNs benchmark.

Machine Learning & Uncertainty Quantification: Application in Building Energy Consumption

“…Inductive Bias [9], Temperature-scaling Calibration [76], Inhibited Softmax [17], Radial Basis Function Networks (DUQ) [77] (Regression) Quantile Loss Function [78], Virtual Residual pretraining [79], Interval NNs [80] External Gradient Metrics [81][82] [83], Additional NN for Uncertainty [84][42], Spectral-Normalized Gaussian Process (SNGP) [85] Hybrid Softmax & Feature Space Regularization (DDU) [86] Ensembles Deep Ensembles Automated deep ensemble with uncertainty quantification (AutoDEUQ) [87], Combination of Base and Meta model [55], (BNN Ensembles) Bayesian Ensembling [88][89], Approximately Bayesian Ensembling [90] Set-based Conformal Prediction Deep Conformal Prediction [92], Inductive Conformal Prediction [93], Conformal in NN [94], Conformal Prediction in CNN [95] Credal Sets Credal UQ [96], Credal Semi-Supervised Learning [97], Credal Bayesian Neural Network [98] Other Literature Test-time augmentation methods [99][100], Epistemic Neural Networks [56] Table 2) Uncertainty Quantification Techniques and Relevant Literature…”

“…Moreover, the posterior distribution of Standard BNNs doesn't allow for the capturing of epistemic uncertainty for each sample [7]. In more recent literature focusing on uncertainty calibrations, BNNs have been identified as providing overconfident and mis-calibrated uncertainty estimations for OOD samples [98].…”

“…Set-Based techniques characterize predictive uncertainty using sets (in contrast to single-point estimates). This approach can be deemed a response to the issue that predictive uncertainty may not always be adequately represented by a single distribution (or single prediction interval), but rather we need to account for multiple plausible outcomes [98]. Since we focus primarily on DL, we limit our discussion of set-based techniques to conformal prediction and credal sets.…”

“…Recent research has focused on the connection between machine learning and credal sets, providing an overview of uncertainty measures and their empirical comparison [96]. [98] proposes Credal Bayesian Deep Learning (CBDL), which utilizes credal sets of likelihood functions and priors. [96], [97], and [98] provide more information for credal sets in machine learning.…”

“…[98] proposes Credal Bayesian Deep Learning (CBDL), which utilizes credal sets of likelihood functions and priors. [96], [97], and [98] provide more information for credal sets in machine learning.…”