HS-Nets: Estimating Human Body Shape from Silhouettes with Convolutional Neural Networks

Dibra, Endri; Jain, Himanshu; Öztireli, Cengiz; Ziegler, Remo; Groß, Markus

doi:10.1109/3dv.2016.19

Cited by 83 publications

(92 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This optimization process is usually initialised with an estimate of the human pose supplied by the user [11] or automatically obtained through a detector [2,20,29] or inertial sensors [42]. Instead of optimizing mesh and skeleton parameters, recent approaches proposed to train neural networks that directly predict 3D shape and skeleton configurations given a monocular RGB video [37], multiple silhouettes [6] or a single image [18,25,28]. Recently, BodyNet [38] was proposed to infer the volumetric body shape through the generation of likelihoods on the 3D occupancy grid of a person from a single image.…”

Section: Related Workmentioning

confidence: 99%

Moulding Humans: Non-Parametric 3D Human Shape Estimation From Single Images

Gabeur

Franco

Martín

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

173

111

View full text Add to dashboard Cite

In this paper, we tackle the problem of 3D human shape estimation from single RGB images. While the recent progress in convolutional neural networks has allowed impressive results for 3D human pose estimation, estimating the full 3D shape of a person is still an open issue. Model-based approaches can output precise meshes of naked under-cloth human bodies but fail to estimate details and un-modelled elements such as hair or clothing. On the other hand, non-parametric volumetric approaches can potentially estimate complete shapes but, in practice, they are limited by the resolution of the output grid and cannot produce detailed estimates. In this work, we propose a non-parametric approach that employs a double depth map to represent the 3D shape of a person: a visible depth map and a "hidden" depth map are estimated and combined, to reconstruct the human 3D shape as done with a "mould". This representation through 2D depth maps allows a higher resolution output with a much lower dimension than voxel-based volumetric representations. Additionally, our fully derivable depth-based model allows us to efficiently incorporate a discriminator in an adversarial fashion to improve the accuracy and "humanness" of the 3D output. We train and quantitatively validate our approach on SURREAL and on 3D-HUMANS, a new photorealistic dataset made of semi-synthetic in-house images annotated with 3D ground truth surfaces.

show abstract

Section: Related Workmentioning

confidence: 99%

Moulding Humans: Non-Parametric 3D Human Shape Estimation From Single Images

Gabeur

Franco

Martín

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

173

111

View full text Add to dashboard Cite

show abstract

“…Dibra et al [106] used an encoder followed by three fully connected layers which regress the SCAPE parameters from one or multiple silhouette images. Later, Dibra et al [107] first learn a common embedding of 2D silhouettes and 3D human body shapes (see Section 7.3.1).…”

Section: D Human Body Reconstructionmentioning

confidence: 99%

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Han

Laga

Bennamoun

2021

IEEE Trans. Pattern Anal. Mach. Intell.

321

150

View full text Add to dashboard Cite

3D reconstruction is a longstanding ill-posed problem, which has been explored for decades by the computer vision, computer graphics, and machine learning communities. Since 2015, image-based 3D reconstruction using convolutional neural networks (CNN) has attracted increasing interest and demonstrated an impressive performance. Given this new era of rapid evolution, this article provides a comprehensive survey of the recent developments in this field. We focus on the works which use deep learning techniques to estimate the 3D shape of generic objects either from a single or multiple RGB images. We organize the literature based on the shape representations, the network architectures, and the training mechanisms they use. While this survey is intended for methods which reconstruct generic objects, we also review some of the recent works which focus on specific object classes such as human body shapes and faces. We provide an analysis and comparison of the performance of some key papers, summarize some of the open problems in this field, and discuss promising directions for future research.

show abstract

“…Recovering 3D human shape from a single image is a challenging problem and has drawn much attention in recent years. A large number of approaches [8,5,6,32,17,33,23,16,21] have been proposed in which the human body shapes get reconstructed by predicting the parameters of a statistical body shape model, such as SMPL [20] and SCAPE [3]. The parametric shape is of low-fidelity, and unable to capture clothing details.…”

Section: Introductionmentioning

confidence: 99%

Detailed Human Shape Estimation From a Single Image by Hierarchical Mesh Deformation

Zhu

Zuo

Wang

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

162

118

View full text Add to dashboard Cite

This paper presents a novel framework to recover detailed human body shapes from a single image. It is a challenging task due to factors such as variations in human shapes, body poses, and viewpoints. Prior methods typically attempt to recover the human body shape using a parametric based template that lacks the surface details. As such the resulting body shape appears to be without clothing. In this paper, we propose a novel learningbased framework that combines the robustness of parametric model with the flexibility of free-form 3D deformation. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation (HMD) framework, utilizing the constraints from body joints, silhouettes, and per-pixel shading information. We are able to restore detailed human body shapes beyond skinned models. Experiments demonstrate that our method has outperformed previous state-of-the-art approaches, achieving better accuracy in terms of both 2D IoU number and 3D metric distance. The code is available in https://github.com/ zhuhao-nju/hmd.git.

show abstract

HS-Nets: Estimating Human Body Shape from Silhouettes with Convolutional Neural Networks

Cited by 83 publications

References 40 publications

Moulding Humans: Non-Parametric 3D Human Shape Estimation From Single Images

Moulding Humans: Non-Parametric 3D Human Shape Estimation From Single Images

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Detailed Human Shape Estimation From a Single Image by Hierarchical Mesh Deformation

Contact Info

Product

Resources

About