A Semi-supervised Deep Generative Model for Human Body Analysis

de, Rodrigo; Ghosh, Arnab; Ajanthan, Thalaiyasingam; Mikšík, Ondřej; Siddharth, N.; Torr, Philip H. S.

doi:10.1007/978-3-030-11012-3_38

Cited by 7 publications

(11 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such labels can be difficult to obtain and are often not defined for continuous variables such as hand pose and viewpoint. In [4,17], a conditional dependency structure is proposed to train disentangled representations for a semi-supervised learning. The work of [4] resembles ours in the sense that they also disentangle pose from appearance; however, their conditional dependency structure is sensitive to the number of factors.…”

Section: Disentangled Representationsmentioning

confidence: 99%

“…In [4,17], a conditional dependency structure is proposed to train disentangled representations for a semi-supervised learning. The work of [4] resembles ours in the sense that they also disentangle pose from appearance; however, their conditional dependency structure is sensitive to the number of factors. As the number of factors grows, the complexity of the network structure increases exponentially.…”

Section: Disentangled Representationsmentioning

confidence: 99%

“…Additional z u : When learning a latent variable model, many latent factors may be very difficult to associate independently with an observation (label), e.g. the style of handwritten digits, or the background content in an RGB image [4,13,2]. Nevertheless, we may still want to disentangle such factors from those which can be associated independently.…”

Section: Disentangled Vaementioning

confidence: 99%

“…A recent work combining VAEs and GANs [4] introduced a conditional dependency structure to learn image backgrounds and demonstrated the possibility of transferring body poses onto different images. Inspired by this work, we would like to learn a similar latent representation that can disentangle the different factors that influence how hands may appear visually, i.e.…”

Section: Introductionmentioning

confidence: 99%

“…Note that in the above ELBO, z u is encoded from x by q φu instead of being specified by some observed label u, as was done previously in [13,2,4]. After this modified disentangling step, we can apply the same embedding step in Eq.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Disentangling Latent Hands for Image Synthesis and Pose Estimation

Yang

Yao

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

125

View full text Add to dashboard Cite

Hand image synthesis and pose estimation from RGB images are both highly challenging tasks due to the large discrepancy between factors of variation ranging from image background content to camera viewpoint. To better analyze these factors of variation, we propose the use of disentangled representations and a disentangled variational autoencoder (dVAE) that allows for specific sampling and inference of these factors. The derived objective from the variational lower bound as well as the proposed training strategy are highly flexible, allowing us to handle crossmodal encoders and decoders as well as semi-supervised learning scenarios. Experiments show that our dVAE can synthesize highly realistic images of the hand specifiable by both pose and image background content and also estimate 3D hand poses from RGB images with accuracy competitive with state-of-the-art on two public benchmarks.

show abstract

Section: Disentangled Representationsmentioning

confidence: 99%

Section: Disentangled Representationsmentioning

confidence: 99%

Section: Disentangled Vaementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Disentangling Latent Hands for Image Synthesis and Pose Estimation

Yang

Yao

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

125

View full text Add to dashboard Cite

show abstract

Human Motion Transfer from Poses in the Wild

Ren

Chai

Tulyakov

et al. 2020

Computer Vision – ECCV 2020 Workshops

View full text Add to dashboard Cite

DGPose: Deep Generative Models for Human Body Analysis

Ghosh

Ajanthan

et al. 2020

Int J Comput Vis

Self Cite

View full text Add to dashboard Cite

Deep generative modelling for human body analysis is an emerging problem with many interesting applications. However, the latent space learned by such approaches is typically not interpretable, resulting in less flexibility. In this work, we present deep generative models for human body analysis in which the body pose and the visual appearance are disentangled. Such a disentanglement allows independent manipulation of pose and appearance, and hence enables applications such as pose-transfer without specific training for such a task. Our proposed models, the Conditional-DGPose and the Semi-DGPose, have different characteristics. In the first, body pose labels are taken as conditioners, from a fully-supervised training set. In the second, our structured semi-supervised approach allows for pose estimation to be performed by the model itself and relaxes the need for labelled data. Therefore, the Semi-DGPose aims for the joint understanding and generation of people in images. It is not only capable of mapping images to interpretable latent representations but also able to map these representations back to the image space. We compare our models with relevant baselines, the ClothNet-Body and the Pose Guided Person Generation networks, demonstrating their merits on the Human3.6M, ChictopiaPlus and DeepFashion benchmarks.

show abstract

A Semi-supervised Deep Generative Model for Human Body Analysis

Cited by 7 publications

References 24 publications

Disentangling Latent Hands for Image Synthesis and Pose Estimation

Disentangling Latent Hands for Image Synthesis and Pose Estimation

Human Motion Transfer from Poses in the Wild

DGPose: Deep Generative Models for Human Body Analysis

Contact Info

Product

Resources

About