An adaptive Gaussian mixture method for nonlinear uncertainty propagation in neural networks

Zhang, Bin; Shin, Yung C.

doi:10.1016/j.neucom.2021.06.007

Cited by 10 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gaussian Mixture Models Dual-supervised mixture of Gaussian Mixture Models [57], Gaussian Mixture Model for uncertainty propagation [58] BNN…”

Section: Bayesian Methodsmentioning

confidence: 99%

“…Their results show improvements in misclassification detection, as well as OOD detection. [58] studies the uncertainty propagation in neural networks with an adaptive Gaussian mixture method. They provide an adaptive Gaussian mixture scheme for highly accurate and faithful representation of the uncertainty, and computationally efficient estimation of uncertainty propagation.…”

Section: Bayesian Methods and Bayesian Neural Networkmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning & Uncertainty Quantification: Application in Building Energy Consumption

Fakour

Parhizkar

Ramezani

et al. 2022

2022 Annual Reliability and Maintainability Symposium (RAMS)

View full text Add to dashboard Cite

The adaptation and use of Machine Learning (ML) in our daily lives has led to concerns in lack of transparency, privacy, reliability, among others. As a result, we are seeing research in niche areas such as interpretability, causality, bias and fairness, and reliability. In this survey paper, we focus on a critical concern for adaptation of ML in risksensitive applications, namely understanding and quantifying uncertainty. Our paper approaches this topic in a structured way, providing a review of the literature in the various facets that uncertainty is enveloped in the ML process. We begin by defining uncertainty and its categories (e.g., aleatoric and epistemic), understanding sources of uncertainty (e.g., data and model), and how uncertainty can be assessed in terms of uncertainty quantification techniques (Ensembles, Bayesian Neural Networks, etc.). As part of our assessment and understanding of uncertainty in the ML realm, we cover metrics for uncertainty quantification for a single sample, dataset, and metrics for accuracy of the uncertainty estimation itself. This is followed by discussions on calibration (model and uncertainty), and decision making under uncertainty. Thus, we provide a more complete treatment of uncertainty: from the sources of uncertainty to the decision-making process. We have focused the review of uncertainty quantification methods on Deep Learning (DL), while providing the necessary background for uncertainty discussion within ML in general. Key contributions in this review are broadening the scope of uncertainty discussion, as well as an updated review of uncertainty quantification methods in DL.

show abstract

“…Gaussian Mixture Models Dual-supervised mixture of Gaussian Mixture Models [57], Gaussian Mixture Model for uncertainty propagation [58] BNN…”

Section: Bayesian Methodsmentioning

confidence: 99%

Section: Bayesian Methods and Bayesian Neural Networkmentioning

confidence: 99%

Machine Learning & Uncertainty Quantification: Application in Building Energy Consumption

Fakour

Parhizkar

Ramezani

et al. 2022

2022 Annual Reliability and Maintainability Symposium (RAMS)

View full text Add to dashboard Cite

show abstract

“…Julier and Uhlmann 1997) is applied. Zhang and Shin (2021) approximate input PDs with Gaussian Mixture Models (GMMs) and demonstrate their propagation through one activation layer. Their main idea is that GMMs with a sufficiently high number of components can approximate arbitrary PDs well.…”

Section: Related Workmentioning

confidence: 99%

Piecewise Linear Transformation – Propagating Aleatoric Uncertainty in Neural Networks

Krapf,

Hagn,

Miethaner

et al. 2024

AAAI

View full text Add to dashboard Cite

Real-world data typically exhibit aleatoric uncertainty which has to be considered during data-driven decision-making to assess the confidence of the decision provided by machine learning models. To propagate aleatoric uncertainty represented by probability distributions (PDs) through neural networks (NNs), both sampling-based and function approximation-based methods have been proposed. However, these methods suffer from significant approximation errors and are not able to accurately represent predictive uncertainty in the NN output. In this paper, we present a novel method, Piecewise Linear Transformation (PLT), for propagating PDs through NNs with piecewise linear activation functions (e.g., ReLU NNs). PLT does not require sampling or specific assumptions about the PDs. Instead, it harnesses the piecewise linear structure of such NNs to determine the propagated PD in the output space. In this way, PLT supports the accurate quantification of predictive uncertainty based on the criterion exactness of the propagated PD. We assess this exactness in theory by showing error bounds for our propagated PD. Further, our experimental evaluation validates that PLT outperforms competing methods on publicly available real-world classification and regression datasets regarding exactness. Thus, the PDs propagated by PLT allow to assess the uncertainty of the provided decisions, offering valuable support.

show abstract

Enhanced Gaussian-mixture-model-based nonlinear probabilistic uncertainty propagation using Gaussian splitting approach

Chen,

Zhang,

et al. 2024

Struct Multidisc Optim

View full text Add to dashboard Cite

An adaptive Gaussian mixture method for nonlinear uncertainty propagation in neural networks

Cited by 10 publications

References 27 publications

Machine Learning & Uncertainty Quantification: Application in Building Energy Consumption

Machine Learning & Uncertainty Quantification: Application in Building Energy Consumption

Piecewise Linear Transformation – Propagating Aleatoric Uncertainty in Neural Networks

Enhanced Gaussian-mixture-model-based nonlinear probabilistic uncertainty propagation using Gaussian splitting approach

Contact Info

Product

Resources

About