Multi-view Consistency Loss for Improved Single-Image 3D Reconstruction of Clothed People

Caliskan, Akin; Mustafa, Armin; Imre, Evren; Hilton, Adrian

doi:10.1007/978-3-030-69525-5_5

Cited by 26 publications

(20 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However all these methods reconstruct a single person with a limited range of pose and require full visibility without occlusions. [8] exploits multi-views during training to estimate 3D human shape from a single image in a wide variety of poses. This method uses a voxel representation which results in a quantized output mesh with low resolution and less accurate reconstruction.…”

Section: Single Person 3d Shape Estimationmentioning

confidence: 99%

“…In this paper, we introduce a method that exploits the advantages of both volumetric voxel and implicit representations by obtaining an initial voxelized mesh of a person which is refined using an implicit function. The proposed method is trained on multiple views [8] to handle partially occluded people in a wide variety of poses.…”

Section: Single Person 3d Shape Estimationmentioning

confidence: 99%

“…3: For each human instance an implicit 3D surface is estimated using an intermediate volumetric voxel-based representation to allow reconstructions from a wide range of poses. The intermediate representation is obtained using multi-view voxel based 3D shape estimation, inspired by [8]. [8] is trained on multiple views for a more accurate reconstruction from a single image.…”

Section: Overviewmentioning

confidence: 99%

“…The intermediate representation is obtained using multi-view voxel based 3D shape estimation, inspired by [8]. [8] is trained on multiple views for a more accurate reconstruction from a single image. A mutli-view occlusion silhouette loss is added to improve the 3D output [45].…”

Section: Overviewmentioning

confidence: 99%

“…The volumetric methods work for a wide variety of poses, clothing and hair, however the output 3D is low resolution which lacks surface details. Implicit representation gives a highly detailed surface, but is limited to restricted poses and clothing [15,8]. In this paper we combine volumetric and implicit representations to benefit from advantages of both, as shown in Fig.…”

Section: Implicit 3d Reconstructionmentioning

confidence: 99%

See 4 more Smart Citations

Multi-person Implicit Reconstruction from a Single Image

Mustafa¹,

Caliskan²,

Hilton³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We present a new end-to-end learning framework to obtain detailed and spatially coherent reconstructions of multiple people from a single image. Existing multi-person methods suffer from two main drawbacks: they are often model-based and therefore cannot capture accurate 3D models of people with loose clothing and hair; or they require manual intervention to resolve occlusions or interactions. Our method addresses both limitations by introducing the first end-to-end learning approach to perform modelfree implicit reconstruction for realistic 3D capture of multiple clothed people in arbitrary poses (with occlusions) from a single image. Our network simultaneously estimates the 3D geometry of each person and their 6DOF spatial locations, to obtain a coherent multi-human reconstruction. In addition, we introduce a new synthetic dataset that depicts images with a varying number of inter-occluded humans and a variety of clothing and hair styles. We demonstrate robust, high-resolution reconstructions on images of multiple humans with complex occlusions, loose clothing and a large variety of poses and scenes. Our quantitative evaluation on both synthetic and real world datasets demonstrates state-of-the-art performance with significant improvements in the accuracy and completeness of the reconstructions over competing approaches.

show abstract

Section: Single Person 3d Shape Estimationmentioning

confidence: 99%

Section: Single Person 3d Shape Estimationmentioning

confidence: 99%

Section: Overviewmentioning

confidence: 99%

Section: Overviewmentioning

confidence: 99%

Section: Implicit 3d Reconstructionmentioning

confidence: 99%

See 3 more Smart Citations

Multi-person Implicit Reconstruction from a Single Image

Mustafa¹,

Caliskan²,

Hilton³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Stable Single-View 3D Human Digitization via Explicit Geometric Field with Semantic Guidance

Liu,

Remagnino

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Garment Ideation: Iterative View-Aware Sketch-Based Garment Modeling

Chowdhury

Wang²,

Ceylan³

et al. 2022

2022 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

Designing real and virtual garments is becoming extremely demanding with rapidly changing fashion trends and increasing need for synthesizing realistically dressed digital humans for various applications. However, traditionally designing real and virtual garments has been time-consuming. Sketch based modeling aims to bring the ease and immediacy of drawing to the 3D world thereby motivating faster iterations. We propose a novel sketchbased garment modeling framework that is specifically targeted to synchronize with the iterative process of garment ideation, e.g., adding or removing details from different views in each iteration. At the core of our learning based approach is a view-aware feature aggregation module that fuses the features from the latest sketch with the thus far aggregated features to effective refine the generated 3D shape. We evaluate our approach on a wide variety of garment types and iterative refinement scenarios. We also provide comparisons to alternative feature aggregation methods and demonstrate favorable results. The code is available at https://github.com/pinakinathc/ multiviewsketch-garment.

show abstract

Multi-view Consistency Loss for Improved Single-Image 3D Reconstruction of Clothed People

Cited by 26 publications

References 39 publications

Multi-person Implicit Reconstruction from a Single Image

Multi-person Implicit Reconstruction from a Single Image

Stable Single-View 3D Human Digitization via Explicit Geometric Field with Semantic Guidance

Garment Ideation: Iterative View-Aware Sketch-Based Garment Modeling

Contact Info

Product

Resources

About