A Mathematical Analysis of Learning Loss for Active Learning in Regression

Shukla, Megh; Ahmed, Shuaib

doi:10.1109/cvprw53098.2021.00370

Cited by 11 publications

(13 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Quantitative analysis. We use the same active learning experiment setup as in [8,47,48,57]. We conduct our experiments on human pose using two single person datasets: MPII [2] and LSP/LSPET [22,23].…”

Section: Methodsmentioning

confidence: 99%

“…Bayesian Neural Networks have been traditionally used to estimate uncertainty; however recent approaches [47,53] explore computing uncertainty using a single forward pass through the network. Empirical approach to estimate ambiguity of the model include Learning Loss [48,57] which similar to our approach uses an auxiliary neural network to predict the 'loss' for an unlabelled image. Approaches that measure model change include expected gradient length [45], which uses the model's gradient as a directly proportionate measure of informativeness.…”

Section: Related Workmentioning

confidence: 99%

“…With VL4Pose, we provide a mathematical framework which not only incorporates heatmap ambiguity but also models spatial relationships between joints. Learning loss [48,57] explored an idea parallel to out-of-distribution detection and identified which images were the most difficult for the model to learn. However, the method has limited ability in identifying out-of-distribution samples.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation

Shukla¹,

Roy²,

Singh³

et al. 2022

Preprint

View full text Add to dashboard Cite

Advances in computing have enabled widespread access to pose estimation, creating new sources of data streams. Unlike mock set-ups for data collection, tapping into these data streams through on-device active learning allows us to directly sample from the real world to improve the spread of the training distribution. However, on-device computing power is limited, implying that any candidate active learning algorithm should have a low compute footprint while also being reliable. Although multiple algorithms cater to pose estimation, they either use extensive compute to power state-of-the-art results or are not competitive in low-resource settings. We address this limitation with VL4Pose (Visual Likelihood For Pose Estimation), a first principles approach for active learning through out-of-distribution detection. We begin with a simple premise: pose estimators often predict incoherent 'poses' for out-of-distribution samples. Hence, can we identify a distribution of poses the model has been trained on, to identify incoherent poses the model is unsure of? Our solution involves modelling the pose through a simple parametric Bayesian network trained via maximum likelihood estimation. Therefore, poses incurring a low likelihood within our framework are out-of-distribution samples making them suitable candidates for annotation. We also observe two useful side-outcomes: VL4Pose in-principle yields better uncertainty estimates by unifying joint and pose level ambiguity, as well as the unintentional but welcome ability of VL4Pose to perform pose refinement in limited scenarios. We perform qualitative and quantitative experiments on three datasets: MPII, LSP and ICVL, spanning human and hand pose estimation. Finally, we note that VL4Pose is simple, computationally inexpensive and competitive, making it suitable for challenging tasks such as on-device active learning.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation

Shukla¹,

Roy²,

Singh³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Active learning (AL), which proactively selects the most informative unlabeled images to annotate, is one promising solution to this problem. Recent active learning-based pose estimation frameworks [38,58,4,5,22] can be categorized into uncertainty-based or distribution-based methods. The uncertainty-based methods [22,58,38] query annotations for the samples with the lowest confidence scores.…”

Section: Introductionmentioning

confidence: 99%

“…Recent active learning-based pose estimation frameworks [38,58,4,5,22] can be categorized into uncertainty-based or distribution-based methods. The uncertainty-based methods [22,58,38] query annotations for the samples with the lowest confidence scores. However, as shown in [24], neural networks tend to be overconfident with unfamiliar samples, leading to overestimated model performance and therefore lowering the labeling efficiency.…”

Section: Introductionmentioning

confidence: 99%

Meta Agent Teaming Active Learning for Pose Estimation

Gong

Fan

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

The existing pose estimation approaches often require a large number of annotated images to attain good estimation performance, which are laborious to acquire. To reduce the human efforts on pose annotations, we propose a novel Meta Agent Teaming Active Learning (MATAL) framework to actively select and label informative images for effective learning. Our MATAL formulates the image selection procedure as a Markov Decision Process and learns an optimal sampling policy that directly maximizes the performance of the pose estimator based on the reward. Our framework consists of a novel state-action representation as well as a multi-agent team to enable batch sampling in the active learning procedure. The framework could be effectively optimized via Meta-Optimization to accelerate the adaptation to the gradually expanded labeled data during deployment. Finally, we show experimental results on both human hand and body pose estimation benchmark datasets and demonstrate that our method significantly outperforms all baselines continuously under the same amount of annotation budget. Moreover, to obtain similar pose estimation accuracy, our MATAL framework can save around 40% labeling efforts on average compared to state-of-the-art active learning frameworks.

show abstract