Deep Sub-Ensembles for Fast Uncertainty Estimation in Image Classification

Valdenegro-Toro, Matías

doi:10.48550/arxiv.1910.08168

Cited by 8 publications

(20 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ensembles emerged as another class of methods for uncertainty estimation [30,49,52]. They are, in fact, a form of Bayesian mixture [53].…”

Section: Related Workmentioning

confidence: 99%

“…While ensembles can provide effective PV scores, they increase both the training and deployment complexities, and may be impractical in some real large-scale systems. Researchers have proposed various methods to reduce computational cost [6,20,35,49,52]. For example, [49] proposed to ensemble only the last several layers to approximate a deep model ensemble, and [52] proposed BatchEnsemble to reduce ensemble complexity by sharing weights among ensemble members.…”

Section: Related Workmentioning

confidence: 99%

“…Many researchers have studied reducing the cost to estimate ensemble based prediction variation [1,6,34,49,52]. For example, [6] demonstrated that it is possible to use neuron activation strength features to infer ensemble prediction variation.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Dropout Prediction Uncertainty Estimation Using Neuron Activation Strength

Yu¹,

Chen²,

Lin³

et al. 2021

Preprint

View full text Add to dashboard Cite

It is well-known that deep neural networks generate different predictions even given the same model configuration and training dataset. It thus becomes more and more important to study prediction variation, the variation of the predictions on a given input example, in neural network models. Dropout has been commonly used in various applications to quantify prediction variations. However, using dropout in practice can be expensive as it requires running dropout inferences many times to estimate prediction variation.We study how to estimate dropout prediction variation in a resource-efficient manner. We demonstrate that we can use neuron activation strengths to estimate dropout prediction variation under different dropout settings and on a variety of tasks using three large datasets, MovieLens, Criteo, and EMNIST. Our approach provides an inference-once alternative to estimate dropout prediction variation as an auxiliary task. Moreover, we demonstrate that using activation features from a subset of the neural network layers can be sufficient to achieve variation estimation performance almost comparable to that of using activation features from all layers, thus reducing resources even further for variation estimation.

show abstract

“…Ensembles emerged as another class of methods for uncertainty estimation [30,49,52]. They are, in fact, a form of Bayesian mixture [53].…”

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Dropout Prediction Uncertainty Estimation Using Neuron Activation Strength

Yu¹,

Chen²,

Lin³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…To better estimate the network uncertainty, many methods were proposed in recent years. Based on the number and the nature of the used DNNs, we can classify them into the single deterministic [13][14][15] , Bayesian 11,16,17 , ensemble [18][19][20] , and test-time augmentation categories [21][22][23][24] . Among these classes, variational Bayesian inference is a commonly used approach.…”

Section: Introductionmentioning

confidence: 99%

SoftDropConnect (SDC) -- Effective and Efficient Quantification of the Network Uncertainty in Deep MR Image Analysis

Lyu¹,

Whitlow²,

Wang³

2022

Preprint

View full text Add to dashboard Cite

Recently, deep learning has achieved remarkable successes in medical image analysis. Although deep neural networks generate clinically important predictions, they have inherent uncertainty. Such uncertainty is a major barrier to report these predictions with confidence. In this paper, we propose a novel yet simple Bayesian inference approach called SoftDropConnect (SDC) to quantify the network uncertainty in medical imaging tasks with gliomas segmentation and metastases classification as initial examples. Our key idea is that during training and testing SDC modulates network parameters continuously so as to allow affected information processing channels still in operation, instead of disabling them as Dropout or DropConnet does. When compared with three popular Bayesian inference methods including Bayes By Backprop, Dropout, and DropConnect, our SDC method (SDC-W after optimization) outperforms the three competing methods with a substantial margin. Quantitatively, our proposed method generates results with substantially improved prediction accuracy (by 10.0%, 5.4% and 3.7% respectively for segmentation in terms of dice score; by 11.7%, 3.9%, 8.7% on classification in terms of test accuracy) and greatly reduced uncertainty in terms of mutual information (by 64%, 33% and 70% on segmentation; 98%, 88%, and 88% on classification). Our approach promises to deliver better diagnostic performance and make medical AI imaging more explainable and trustworthy.

show abstract

“…Recent studies have shown that the ensemble approaches are effective to improve accuracy and estimate model uncertainty [2,46,12]. In ensemble studies [28,18], M different models which have different weight parameters could be constructed and the mean of the predictions is used for the final prediction.…”

Section: Introductionmentioning

confidence: 99%

Efficient Ensemble Model Generation for Uncertainty Estimation with Bayesian Approximation in Segmentation

Lee¹,

Kim²,

Lee³

et al. 2020

Preprint

View full text Add to dashboard Cite

Recent studies have shown that ensemble approaches could not only improve accuracy and but also estimate model uncertainty in deep learning. However, it requires a large number of parameters according to the increase of ensemble models for better prediction and uncertainty estimation. To address this issue, a generic and efficient segmentation framework to construct ensemble segmentation models is devised in this paper. In the proposed method, ensemble models can be efficiently generated by using the stochastic layer selection method. The ensemble models are trained to estimate uncertainty through Bayesian approximation. Moreover, to overcome its limitation from uncertain instances, we devise a new pixel-wise uncertainty loss, which improves the predictive performance. To evaluate our method, comprehensive and comparative experiments have been conducted on two datasets. Experimental results show that the proposed method could provide useful uncertainty information by Bayesian approximation with the efficient ensemble model generation and improve the predictive performance.

show abstract

Deep Sub-Ensembles for Fast Uncertainty Estimation in Image Classification

Cited by 8 publications

References 0 publications

Dropout Prediction Uncertainty Estimation Using Neuron Activation Strength

Dropout Prediction Uncertainty Estimation Using Neuron Activation Strength

SoftDropConnect (SDC) -- Effective and Efficient Quantification of the Network Uncertainty in Deep MR Image Analysis

Efficient Ensemble Model Generation for Uncertainty Estimation with Bayesian Approximation in Segmentation

Contact Info

Product

Resources

About