Lifting AutoEncoders: Unsupervised Learning of a Fully-Disentangled 3D Morphable Model using Deep Non-Rigid Structure from Motion

Sahasrabudhe, Mihir; Shu, Zhixin; Bartrum, Edward; Guler, Riza; Samaras, Dimitris; Kokkinos, Iasonas

doi:10.48550/arxiv.1904.11960

Cited by 2 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, Mandikal et al [23] used a hierarchical model to generate a point set prediction of an input image by using three stages. In recent years, mesh-based methods are proposed which perform reconstruction by warping a simple shape [44,36,16]. AtlasNet [8] obtained impressive results in meshbased object generation by warping and combining primitive surface patches.…”

Section: D Shape Representationmentioning

confidence: 99%

Self-Supervised 2D Image to 3D Shape Translation with Disentangled Representations

Kaya¹,

Timofte²

2020

Preprint

View full text Add to dashboard Cite

We present a framework to translate between 2D image views and 3D object shapes. Recent progress in deep learning enabled us to learn structure-aware representations from a scene. However, the existing literature assumes that pairs of images and 3D shapes are available for training in full supervision. In this paper, we propose SIST, a Self-supervised Image to Shape Translation framework that fulfills three tasks: (i) reconstructing the 3D shape from a single image; (ii) learning disentangled representations for shape, appearance and viewpoint; and (iii) generating a realistic RGB image from these independent factors. In contrast to the existing approaches, our method does not require image-shape pairs for training. Instead, it uses unpaired image and shape datasets from the same object class and jointly trains image generator and shape reconstruction networks. Our translation method achieves promising results, comparable in quantitative and qualitative terms to the state-of-the-art achieved by fully-supervised methods 1 .

show abstract

Section: D Shape Representationmentioning

confidence: 99%

Self-Supervised 2D Image to 3D Shape Translation with Disentangled Representations

Kaya¹,

Timofte²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The method of [33] learns 3D landmarks detectors from images related by known 3D transformations, again by applying equivariance. Finally, it is worth mentioning other recent works in generative models that, rather than directly predicting a set of landmarks, focus on disentangling shape and appearance in an autoencoder framework [31,30,19]. In [31,30], the learning is formulated as an autoencoder that generates a dense warp map and the appearance in a reference frame.…”

Section: Related Workmentioning

confidence: 99%

“…Finally, it is worth mentioning other recent works in generative models that, rather than directly predicting a set of landmarks, focus on disentangling shape and appearance in an autoencoder framework [31,30,19]. In [31,30], the learning is formulated as an autoencoder that generates a dense warp map and the appearance in a reference frame. While the warps can be mapped to specific keypoints, their discovery is not the target goal.…”

Section: Related Workmentioning

confidence: 99%

Object landmark discovery through unsupervised adaptation

Sanchez,

Tzimiropoulos

2019

Preprint

View full text Add to dashboard Cite

This paper proposes a method to ease the unsupervised learning of object landmark detectors. Similarly to previous methods, our approach is fully unsupervised in a sense that it does not require or make any use of annotated landmarks for the target object category. Contrary to previous works, we do however assume that a landmark detector, which has already learned a structured representation for a given object category in a fully supervised manner, is available. Under this setting, our main idea boils down to adapting the given pre-trained network to the target object categories in a fully unsupervised manner. To this end, our method uses the pre-trained network as a core which remains frozen and does not get updated during training, and learns, in an unsupervised manner, only a projection matrix to perform the adaptation to the target categories. By building upon an existing structured representation learned in a supervised manner, the optimization problem solved by our method is much more constrained with significantly less parameters to learn which seems to be important for the case of unsupervised learning. We show that our method surpasses fully unsupervised techniques trained from scratch as well as a strong baseline based on fine-tuning, and produces state-of-the-art results on several datasets. Code can be found at tiny.cc/GitHub-Unsupervised.

show abstract

Lifting AutoEncoders: Unsupervised Learning of a Fully-Disentangled 3D Morphable Model using Deep Non-Rigid Structure from Motion

Cited by 2 publications

References 0 publications

Self-Supervised 2D Image to 3D Shape Translation with Disentangled Representations

Self-Supervised 2D Image to 3D Shape Translation with Disentangled Representations

Object landmark discovery through unsupervised adaptation

Contact Info

Product

Resources

About