A Field Model for Repairing 3D Shapes

Nguyen, Duc Thanh; Hua, Binh-Son; Tran, Minh-Khoi; Pham, Quang-Hieu; Yeung, Sai-Kit

doi:10.1109/cvpr.2016.612

Cited by 98 publications

(50 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…3D ShapeNets [58] uses a deep belief network to obtain a generative model for a given shape database, and Nguyen et al . [50] extend the method for mesh repairing.…”

Section: Related Workmentioning

confidence: 99%

Learning Shape Priors for Single-View 3D Completion And Reconstruction

Wu¹,

Zhang²,

Zhang³

et al. 2018

Lecture Notes in Computer Science

180

147

View full text Add to dashboard Cite

The problem of single-view 3D shape completion or reconstruction is challenging, because among the many possible shapes that explain an observation, most are implausible and do not correspond to natural objects. Recent research in the field has tackled this problem by exploiting the expressiveness of deep convolutional networks. In fact, there is another level of ambiguity that is often overlooked: among plausible shapes, there are still multiple shapes that fit the 2D image equally well; i.e., the ground truth shape is non-deterministic given a single-view input. Existing fully supervised approaches fail to address this issue, and often produce blurry mean shapes with smooth surfaces but no fine details. In this paper, we propose ShapeHD, pushing the limit of single-view shape completion and reconstruction by integrating deep generative models with adversarially learned shape priors. The learned priors serve as a regularizer, penalizing the model only if its output is unrealistic, not if it deviates from the ground truth. Our design thus overcomes both levels of ambiguity aforementioned. Experiments demonstrate that ShapeHD outperforms state of the art by a large margin in both shape completion and shape reconstruction on multiple real datasets.

show abstract

“…3D ShapeNets [58] uses a deep belief network to obtain a generative model for a given shape database, and Nguyen et al . [50] extend the method for mesh repairing.…”

Section: Related Workmentioning

confidence: 99%

Learning Shape Priors for Single-View 3D Completion And Reconstruction

Wu¹,

Zhang²,

Zhang³

et al. 2018

Lecture Notes in Computer Science

180

147

View full text Add to dashboard Cite

show abstract

“…A set of methods reconstruct the 3D object shape from a single depth image using 3D shape retrieval [Rock et al, 2015], Convolutional Deep Belief Network (CDBN) [Wu et al, 2015;Nguyen et al, 2016], or a 3D Generative Adversarial Networks (GAN) [Yang et al, 2017]. All these methods model the input depth maps and resulting 3D shapes with a 3D volumetric representation.…”

Section: D Object Completionmentioning

confidence: 99%

“…Although these methods can fully exploit the high resolution input to generate detailed segmentation results, they ignore the 3D context information of the scene and thus cannot infer the invisible part of the scene. 3D CNN based methods [Wu et al, 2015;Nguyen et al, 2016;Song and Xiao, 2016; convert input depth maps or point clouds into a volumetric representation and design 3D CNNs for 3D scene segmentation or object completion. However, the high computational and memory cost of the 3D CNN limits their capability for recovering object details.…”

Section: Introductionmentioning

confidence: 99%

View-Volume Network for Semantic Scene Completion from a Single Depth Image

Guo

Tong

2018

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

We introduce a View-Volume convolutional neural network (VVNet) for inferring the occupancy and semantic labels of a volumetric 3D scene from a single depth image. The VVNet concatenates a 2D view CNN and a 3D volume CNN with a differentiable projection layer. Given a single RGBD image, our method extracts the detailed geometric features from the input depth image with a 2D view CNN and then projects the features into a 3D volume according to the input depth map via a projection layer. After that, we learn the 3D context information of the scene with a 3D volume CNN for computing the result volumetric occupancy and semantic labels. With combined 2D and 3D representations, the VVNet efficiently reduces the computational cost, enables feature extraction from multi-channel high resolution inputs, and thus significantly improves the result accuracy. We validate our method and demonstrate its efficiency and effectiveness on both synthetic SUNCG and real NYU dataset.

show abstract

“…In these cases, shapes are usually represented by signed distance functions (SDFs). Nguyen et al (2016) use 3DShapeNets (Wu et al, 2015), a deep belief network trained on occupancy grids, as shape prior. In general, data-driven approaches are applicable to real data assuming knowledge about the object category.…”

Section: D Shape Completion and Single-view 3d Reconstructionmentioning

confidence: 99%

“…Existing approaches to 3D shape completion can be categorized into data-driven and learning-based methods. The former usually rely on learned shape priors and formulate shape completion as an optimization problem over the corresponding (lower-dimensional) latent space (Rock et al, 2015;Haene et al, 2014;Li et al, 2015;Engelmann et al, 2016;Nan et al, 2012;Bao et al, 2013;Dame et al, 2013;Nguyen et al, 2016). These approaches have demonstrated good performance on real data, e.g., on KITTI (Geiger et al, 2012), but are often slow in practice.…”

mentioning

confidence: 99%

Learning 3D Shape Completion Under Weak Supervision

Stutz

Geiger

2018

Int J Comput Vis

View full text Add to dashboard Cite

We address the problem of 3D shape completion from sparse and noisy point clouds, a fundamental problem in computer vision and robotics. Recent approaches are either data-driven or learning-based: Datadriven approaches rely on a shape model whose parameters are optimized to fit the observations; Learningbased approaches, in contrast, avoid the expensive optimization step by learning to directly predict complete shapes from incomplete observations in a fullysupervised setting. However, full supervision is often not available in practice. In this work, we propose a weakly-supervised learning-based approach to 3D shape completion which neither requires slow optimization nor direct supervision. While we also learn a shape prior on synthetic data, we amortize, i.e., learn, maximum likelihood fitting using deep neural networks resulting in efficient shape completion without sacrificing accuracy. On synthetic benchmarks based on ShapeNet (Chang et al, 2015) and ModelNet (Wu et al, 2015) as well as on real robotics data from KITTI (Geiger et al, 2012) and Kinect (Yang et al, 2018), we demonstrate that the proposed amortized maximum likelihood approach is able to compete with the fully supervised baseline of Dai et al (2017) and outperforms the data-driven approach of Engelmann et al (2016), while requiring less supervision and being significantly faster.

show abstract

A Field Model for Repairing 3D Shapes

Cited by 98 publications

References 28 publications

Learning Shape Priors for Single-View 3D Completion And Reconstruction

Learning Shape Priors for Single-View 3D Completion And Reconstruction

View-Volume Network for Semantic Scene Completion from a Single Depth Image

Learning 3D Shape Completion Under Weak Supervision

Contact Info

Product

Resources

About