Bayesian Uncertainty and Expected Gradient Length - Regression: Two Sides Of The Same Coin?

Shukla, Megh

doi:10.1109/wacv51458.2022.00208

Cited by 2 publications

(9 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Uncertainty estimation [10,12,13,16,25,31] provides quantitative measures to model ambiguity in the prediction. Bayesian Neural Networks have been traditionally used to estimate uncertainty; however recent approaches [47,53] explore computing uncertainty using a single forward pass through the network. Empirical approach to estimate ambiguity of the model include Learning Loss [48,57] which similar to our approach uses an auxiliary neural network to predict the 'loss' for an unlabelled image.…”

Section: Related Workmentioning

confidence: 99%

“…Approaches that measure model change include expected gradient length [45], which uses the model's gradient as a directly proportionate measure of informativeness. Application domains [6,19,46,47] of expected gradient length include image and text analysis.…”

Section: Related Workmentioning

confidence: 99%

“…Active Learning for Pose Estimation. Challenges posed by popular human pose architectures (detailed in [8,47]) have lead to the development of new algorithms. Liu and Ferrari [34] proposed an intuition driven framework to model heatmap ambiguity.…”

Section: Related Workmentioning

confidence: 99%

“…Specifically, popular human pose estimation architectures are fully convolutional and directly regress 2D heatmaps [28,58]. This prevents [8,47] the use of many algorithms which rely on ensembles, dropouts or logits. As a result, a new wave of active learning methods [8,14,34,47,57,59] address keypoint estimation.…”

Section: Introductionmentioning

confidence: 99%

“…This prevents [8,47] the use of many algorithms which rely on ensembles, dropouts or logits. As a result, a new wave of active learning methods [8,14,34,47,57,59] address keypoint estimation.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation

Shukla¹,

Roy²,

Singh³

et al. 2022

Preprint

View full text Add to dashboard Cite

Advances in computing have enabled widespread access to pose estimation, creating new sources of data streams. Unlike mock set-ups for data collection, tapping into these data streams through on-device active learning allows us to directly sample from the real world to improve the spread of the training distribution. However, on-device computing power is limited, implying that any candidate active learning algorithm should have a low compute footprint while also being reliable. Although multiple algorithms cater to pose estimation, they either use extensive compute to power state-of-the-art results or are not competitive in low-resource settings. We address this limitation with VL4Pose (Visual Likelihood For Pose Estimation), a first principles approach for active learning through out-of-distribution detection. We begin with a simple premise: pose estimators often predict incoherent 'poses' for out-of-distribution samples. Hence, can we identify a distribution of poses the model has been trained on, to identify incoherent poses the model is unsure of? Our solution involves modelling the pose through a simple parametric Bayesian network trained via maximum likelihood estimation. Therefore, poses incurring a low likelihood within our framework are out-of-distribution samples making them suitable candidates for annotation. We also observe two useful side-outcomes: VL4Pose in-principle yields better uncertainty estimates by unifying joint and pose level ambiguity, as well as the unintentional but welcome ability of VL4Pose to perform pose refinement in limited scenarios. We perform qualitative and quantitative experiments on three datasets: MPII, LSP and ICVL, spanning human and hand pose estimation. Finally, we note that VL4Pose is simple, computationally inexpensive and competitive, making it suitable for challenging tasks such as on-device active learning.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation

Shukla¹,

Roy²,

Singh³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Deep Active Learning by Leveraging Training Dynamics

Wang

Huang

Margenot

et al. 2021

Preprint

View full text Add to dashboard Cite

Active learning theories and methods have been extensively studied in classical statistical learning settings. However, deep active learning, i.e., active learning with deep learning models, is usually based on empirical criteria without solid theoretical justification, thus suffering from heavy doubts when some of those fail to provide benefits in applications. In this paper, by exploring the connection between the generalization performance and the training dynamics, we propose a theory-driven deep active learning method (dynamicAL) which selects samples to maximize training dynamics. In particular, we prove that convergence speed of training and the generalization performance is positively correlated under the ultra-wide condition and show that maximizing the training dynamics leads to a better generalization performance. Further on, to scale up to large deep neural networks and data sets, we introduce two relaxations for the subset selection problem and reduce the time complexity from polynomial to constant. Empirical results show that dynamicAL not only outperforms the other baselines consistently but also scales well on large deep learning models. We hope our work inspires more attempts in bridging the theoretical findings of deep networks and practical impacts in deep active learning applications.

show abstract

Bayesian Uncertainty and Expected Gradient Length - Regression: Two Sides Of The Same Coin?

Cited by 2 publications

References 30 publications

VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation

VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation

Deep Active Learning by Leveraging Training Dynamics

Contact Info

Product

Resources

About