Probabilistic Monocular 3D Human Pose Estimation with Normalizing Flows

Wehrbein, Tom; Rudolph, Marco; Rosenhahn, Bodo; Wandt, Bastian

doi:10.1109/iccv48922.2021.01101

Cited by 92 publications

(17 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sharma et al [55] propose to solve the ill-posed 2D-to-3D lifting problem with CVAE [59] and Ordinal Ranking. Wehrbein et al [67] use Normalizing Flows to model the deterministic 3D-2D projection and solve the ambiguous inverse 2D-3D lifting problem. The major difference between our work and multiple-hypotheses 3D pose estimation is that our model is trained without 3D ground truth.…”

Section: Related Workmentioning

confidence: 99%

“…However, generating multi-hypothesis reconstruction for NRSfM is challenging for several reasons: (1) Without 3D ground-truth as supervision, the ambiguous 2D-to-3D mappings cannot be learned using standard generative models like CVAE [59], Conditional GAN [37] or Normalizing Flow [67]. (2) Multiple hypotheses easily suffer from the decomposition ambiguity of NRSfM [18], i.e.…”

Section: Multiple Hypothesis Reconstruction Network -Overviewmentioning

confidence: 99%

See 1 more Smart Citation

MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Views

Zeng¹,

Yu²,

Miao³

et al. 2022

Preprint

View full text Add to dashboard Cite

We propose MHR-Net, a novel method for recovering Non-Rigid Shapes from Motion (NRSfM). MHR-Net aims to find a set of reasonable reconstructions for a 2D view, and it also selects the most likely reconstruction from the set. To deal with the challenging unsupervised generation of non-rigid shapes, we develop a new Deterministic Basis and Stochastic Deformation scheme in MHR-Net. The non-rigid shape is first expressed as the sum of a coarse shape basis and a flexible shape deformation, then multiple hypotheses are generated with uncertainty modeling of the deformation part. MHR-Net is optimized with reprojection loss on the basis and the best hypothesis. Furthermore, we design a new Procrustean Residual Loss, which reduces the rigid rotations between similar shapes and further improves the performance. Experiments show that MHR-Net achieves state-of-the-art reconstruction accuracy on Human3.6M, SURREAL and 300-VW datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Multiple Hypothesis Reconstruction Network -Overviewmentioning

confidence: 99%

MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Views

Zeng¹,

Yu²,

Miao³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent state-of-the-art methods such as MATAL [14] and UncertainGCN [9] use powerful techniques such as reinforcement learning and graph convolutional networks, however they are computationally very expensive. Uncertainty in 3D human pose [4,44,56] utilizes depth information or stereo images, both of which are not available for general 2D pose estimation. We reserve our discussion on uncertainty in 2D for later.…”

Section: Related Workmentioning

confidence: 99%

VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation

Shukla¹,

Roy²,

Singh³

et al. 2022

Preprint

View full text Add to dashboard Cite

Advances in computing have enabled widespread access to pose estimation, creating new sources of data streams. Unlike mock set-ups for data collection, tapping into these data streams through on-device active learning allows us to directly sample from the real world to improve the spread of the training distribution. However, on-device computing power is limited, implying that any candidate active learning algorithm should have a low compute footprint while also being reliable. Although multiple algorithms cater to pose estimation, they either use extensive compute to power state-of-the-art results or are not competitive in low-resource settings. We address this limitation with VL4Pose (Visual Likelihood For Pose Estimation), a first principles approach for active learning through out-of-distribution detection. We begin with a simple premise: pose estimators often predict incoherent 'poses' for out-of-distribution samples. Hence, can we identify a distribution of poses the model has been trained on, to identify incoherent poses the model is unsure of? Our solution involves modelling the pose through a simple parametric Bayesian network trained via maximum likelihood estimation. Therefore, poses incurring a low likelihood within our framework are out-of-distribution samples making them suitable candidates for annotation. We also observe two useful side-outcomes: VL4Pose in-principle yields better uncertainty estimates by unifying joint and pose level ambiguity, as well as the unintentional but welcome ability of VL4Pose to perform pose refinement in limited scenarios. We perform qualitative and quantitative experiments on three datasets: MPII, LSP and ICVL, spanning human and hand pose estimation. Finally, we note that VL4Pose is simple, computationally inexpensive and competitive, making it suitable for challenging tasks such as on-device active learning.

show abstract

“…A normalizing flow (NF) [28] is a generative model that transforms data into tractable distributions. Unlike conventional neural networks, their mapping is bijective, which allows them to train and evaluate in both directions [39]. The forward pass projects data into a latent space to calculate exact likelihoods for the data given the predefined latent distribution.…”

Section: Normalizing Flowsmentioning

confidence: 99%

Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection

Rudolph¹,

Wehrbein²,

Rosenhahn³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In industrial manufacturing processes, errors frequently occur at unpredictable times and in unknown manifestations. We tackle the problem of automatic defect detection without requiring any image samples of defective parts. Recent works model the distribution of defect-free image data, using either strong statistical priors or overly simplified data representations. In contrast, our approach handles fine-grained representations incorporating the global and local image context while flexibly estimating the density. To this end, we propose a novel fully convolutional cross-scale normalizing flow (CS-Flow) that jointly processes multiple feature maps of different scales. Using normalizing flows to assign meaningful likelihoods to input samples allows for efficient defect detection on image-level. Moreover, due to the preserved spatial arrangement the latent space of the normalizing flow is interpretable which enables to localize defective regions in the image. Our work sets a new stateof-the-art in image-level defect detection on the benchmark datasets Magnetic Tile Defects and MVTec AD showing a 100% AUROC on 4 out of 15 classes.

show abstract

Probabilistic Monocular 3D Human Pose Estimation with Normalizing Flows

Cited by 92 publications

References 32 publications

MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Views

MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Views

VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation

Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection

Contact Info

Product

Resources

About