On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks

Seitzer, Maximilian; Tavakoli, Arash; Antic, Dimitrije; Martius, Georg

doi:10.48550/arxiv.2203.09168

“…44 Mean-variance estimation and similar techniques can be very successful in training models on noisy datasets if the error is a function of the input features, since it allows the model to learn on which datapoints to focus, and which to regard as unreliable. 34,46,[63][64][65] However, it is not amenable to noise that is uniformly distributed over all datapoints or systematic noise that is applied based on external factors not represented in the input features of the training data. For example, if one measurement instrument had increased noise in data collection but the identity of the instrument used in collection was not included in the input features and could not be inferred from the input features, then the systematic noise applied according to the external factor of a faulty instrument would not be distinguishable.…”

Section: Systematic Noisementioning

confidence: 99%

“…Concerns around sub-optimal performance of mean-variance estimation techniques have been recently reported in literature. 65 We therefore recommend that users consider whether there are identifiable sources of systematic noise related to model input features and that they compare performance of a mean-variance estimation model against a simple model.…”

Section: Systematic Noisementioning

confidence: 99%

Characterizing Uncertainty in Machine Learning for Chemistry

Heid

¹

,

McGill

²

,

Vermeire

³

et al. 2023

Preprint

View full text Add to dashboard Cite

Characterizing uncertainty in machine learning models has recently gained interest in the context of machine learning reliability, robustness, safety, and active learning. Here, we separate the total uncertainty into contributions from noise in the data (aleatoric) and shortcomings of the model (epistemic), further dividing epistemic uncertainty into model bias and variance contributions. We systematically address the influence of noise, model bias, and model variance in the context of chemical property predictions, where the diverse nature of target properties and the vast chemical chemical space give rise to many different distinct sources of prediction error. We demonstrate that different sources of error can each be significant in different contexts and must be individually addressed during model development. Through controlled experiments on datasets of molecular properties, we show important trends in model performance associated with the level of noise in the dataset, size of the dataset, model architecture, molecule representation, ensemble size, and dataset splitting. In particular, we show that 1) noise in the test set can limit a model's observed performance when the actual performance is much better, 2) using size-extensive model aggregation structures is crucial for extensive property prediction, 3) ensembling is a reliable tool for uncertainty quantification and improvement specifically for the contribution of model variance, and 4) evaluations of cross-validation models understate their performance. We develop general guidelines on how to improve an underperforming model when falling into different uncertainty contexts.

show abstract

“…44 Mean-variance estimation and similar techniques can be very successful in training models on noisy datasets if the error is a function of the input features, since it allows the model to learn on which datapoints to focus, and which to regard as unreliable. 34,46,[63][64][65] However, it is not amenable to noise that is uniformly distributed over all datapoints or systematic noise that is applied based on external factors not represented in the input features of the training data. For example, if one measurement instrument had increased noise in data collection but the identity of the instrument used in collection was not included in the input features and could not be inferred from the input features, then the systematic noise applied according to the external factor of a faulty instrument would not be distinguishable.…”

Section: Systematic Noisementioning

confidence: 99%

Characterizing Uncertainty in Machine Learning for Chemistry

Heid

¹

,

McGill

²

,

Vermeire

³

et al. 2023

Preprint

View full text Add to dashboard Cite

Characterizing uncertainty in machine learning models has recently gained interest in the context of machine learning reliability, robustness, safety, and active learning. Here, we separate the total uncertainty into contributions from noise in the data (aleatoric) and shortcomings of the model (epistemic), further dividing epistemic uncertainty into model bias and variance contributions. We systematically address the influence of noise, model bias, and model variance in the context of chemical property predictions, where the diverse nature of target properties and the vast chemical chemical space give rise to many different distinct sources of prediction error. We demonstrate that different sources of error can each be significant in different contexts and must be individually addressed during model development. Through controlled experiments on datasets of molecular properties, we show important trends in model performance associated with the level of noise in the dataset, size of the dataset, model architecture, molecule representation, ensemble size, and dataset splitting. In particular, we show that 1) noise in the test set can limit a model's observed performance when the actual performance is much better, 2) using size-extensive model aggregation structures is crucial for extensive property prediction, 3) ensembling is a reliable tool for uncertainty quantification and improvement specifically for the contribution of model variance, and 4) evaluations of cross-validation models understate their performance. We develop general guidelines on how to improve an underperforming model when falling into different uncertainty contexts.

show abstract

“…In general, ( 14) can be replaced by any tractable function that measures the similarity of distributions. For example, when π φ is Gaussian, we can apply the recent proposed β-NLL [41], in which each data point's contribution to the negative log-likelihood loss is weighted by the β-exponentiated variance to improve learning heteroscedastic behavior.…”

Section: Offline Reinforcement Learning With Pessimism-modulated Dyna...mentioning

confidence: 99%

Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Belief

Guo¹,

Shao²,

Geng³

2022

Preprint

View full text Add to dashboard Cite

Model-based offline reinforcement learning (RL) aims to find highly rewarding policy, by leveraging a previously collected static dataset and a dynamics model. While the dynamics model learned through reuse of the static dataset, its generalization ability hopefully promotes policy learning if properly utilized. To that end, several works propose to quantify the uncertainty of predicted dynamics, and explicitly apply it to penalize reward. However, as the dynamics and the reward are intrinsically different factors in context of MDP, characterizing the impact of dynamics uncertainty through reward penalty may incur unexpected tradeoff between model utilization and risk avoidance. In this work, we instead maintain a belief distribution over dynamics, and evaluate/optimize policy through biased sampling from the belief. The sampling procedure, biased towards pessimism, is derived based on an alternating Markov game formulation of offline RL. We formally show that the biased sampling naturally induces an updated dynamics belief with policy-dependent reweighting factor, termed Pessimism-Modulated Dynamics Belief. To improve policy, we devise an iterative regularized policy optimization algorithm for the game, with guarantee of monotonous improvement under certain condition. To make practical, we further devise an offline RL algorithm to approximately find the solution. Empirical results show that the proposed approach achieves state-of-the-art performance on a wide range of benchmark tasks.

show abstract

On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks

Cited by 14 publications

References 0 publications

Characterizing Uncertainty in Machine Learning for Chemistry

Characterizing Uncertainty in Machine Learning for Chemistry

Characterizing Uncertainty in Machine Learning for Chemistry

Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Belief

Contact Info

Product

Resources

About