BaCOUn: Bayesian Classifers with Out-of-Distribution Uncertainty

Guénais, Théo; Vamvourellis, Dimitris; Yacoby, Yaniv; Doshi-Velez, Finale; Pan, Weiwei

doi:10.48550/arxiv.2007.06096

Cited by 3 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For larger datasets, more scalable models such as neural linear model (NLM) are used. 23 Assuming linearity, one may also implement Bayesian linear regression for binary/categorical or the generalized linear model for continuous variables. 2. when L r = ∅.…”

Section: E4 MI Samplingmentioning

confidence: 99%

Assessable and interpretable sensitivity analysis in the pattern graph framework for nonignorable missingness mechanisms

Zamanian,

Ahmidi,

Drton

2023

Statistics in Medicine

View full text Add to dashboard Cite

The pattern graph framework solves a wide range of missing data problems with nonignorable mechanisms. However, it faces two challenges of assessability and interpretability, particularly important in safety‐critical problems such as clinical diagnosis: (i) How can one assess the validity of the framework's a priori assumption and make necessary adjustments to accommodate known information about the problem? (ii) How can one interpret the process of exponential tilting used for sensitivity analysis in the pattern graph framework and choose the tilt perturbations based on meaningful real‐world quantities? In this paper, we introduce Informed Sensitivity Analysis, an extension of the pattern graph framework that enables us to incorporate substantive knowledge about the missingness mechanism into the pattern graph framework. Our extension allows us to examine the validity of assumptions underlying pattern graphs and interpret sensitivity analysis results in terms of realistic problem characteristics. We apply our method to a prevalent nonignorable missing data scenario in clinical research. We validate and compare our method's results of our method with a number of widely‐used missing data methods, including Unweighted CCA, KNN Imputer, MICE, and MissForest. The validation is done using both boot‐strapped simulated experiments as well as real‐world clinical observations in the MIMIC‐III public dataset.

show abstract

Section: E4 MI Samplingmentioning

confidence: 99%

Assessable and interpretable sensitivity analysis in the pattern graph framework for nonignorable missingness mechanisms

Zamanian,

Ahmidi,

Drton

2023

Statistics in Medicine

View full text Add to dashboard Cite

show abstract

“…Diversity based approaches include Core-Set [43] which sequentially selects non-semantically similar points. Uncertainty estimation [10,12,13,16,25,31] provides quantitative measures to model ambiguity in the prediction. Bayesian Neural Networks have been traditionally used to estimate uncertainty; however recent approaches [47,53] explore computing uncertainty using a single forward pass through the network.…”

Section: Related Workmentioning

confidence: 99%

VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation

Shukla¹,

Roy²,

Singh³

et al. 2022

Preprint

View full text Add to dashboard Cite

Advances in computing have enabled widespread access to pose estimation, creating new sources of data streams. Unlike mock set-ups for data collection, tapping into these data streams through on-device active learning allows us to directly sample from the real world to improve the spread of the training distribution. However, on-device computing power is limited, implying that any candidate active learning algorithm should have a low compute footprint while also being reliable. Although multiple algorithms cater to pose estimation, they either use extensive compute to power state-of-the-art results or are not competitive in low-resource settings. We address this limitation with VL4Pose (Visual Likelihood For Pose Estimation), a first principles approach for active learning through out-of-distribution detection. We begin with a simple premise: pose estimators often predict incoherent 'poses' for out-of-distribution samples. Hence, can we identify a distribution of poses the model has been trained on, to identify incoherent poses the model is unsure of? Our solution involves modelling the pose through a simple parametric Bayesian network trained via maximum likelihood estimation. Therefore, poses incurring a low likelihood within our framework are out-of-distribution samples making them suitable candidates for annotation. We also observe two useful side-outcomes: VL4Pose in-principle yields better uncertainty estimates by unifying joint and pose level ambiguity, as well as the unintentional but welcome ability of VL4Pose to perform pose refinement in limited scenarios. We perform qualitative and quantitative experiments on three datasets: MPII, LSP and ICVL, spanning human and hand pose estimation. Finally, we note that VL4Pose is simple, computationally inexpensive and competitive, making it suitable for challenging tasks such as on-device active learning.

show abstract

“…Application examples: Guénais et al [44] proposed a Bayesian framework to obtain reliable uncertainty estimates for deep classifiers. Their approach consists of a plug-in "generator" used to augment the data with an additional class of points on the boundary of the training data, followed by Bayesian inference on top of features trained to distinguish these "out-of-distribution" points.…”

Section: Introductionmentioning

confidence: 99%

Out-of-Distribution (OOD) Detection Based on Deep Learning: A Review

Cui

Wang

2022

Electronics

View full text Add to dashboard Cite

Out-of-Distribution (OOD) detection separates ID (In-Distribution) data and OOD data from input data through a model. This problem has attracted increasing attention in the area of machine learning. OOD detection has achieved good intrusion detection, fraud detection, system health monitoring, sensor network event detection, and ecosystem interference detection. The method based on deep learning is the most studied in OOD detection. In this paper, related basic information on OOD detection based on deep learning is described, and we categorize methods according to the training data. OOD detection is divided into supervised, semisupervised, and unsupervised. Where supervised data are used, the methods are categorized according to technical means: model-based, distance-based, and density-based. Each classification is introduced with background, examples, and applications. In addition, we present the latest applications of OOD detection based on deep learning and the problems and expectations in this field.

show abstract

BaCOUn: Bayesian Classifers with Out-of-Distribution Uncertainty

Cited by 3 publications

References 16 publications

Assessable and interpretable sensitivity analysis in the pattern graph framework for nonignorable missingness mechanisms

Assessable and interpretable sensitivity analysis in the pattern graph framework for nonignorable missingness mechanisms

VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation

Out-of-Distribution (OOD) Detection Based on Deep Learning: A Review

Contact Info

Product

Resources

About