3D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction from a Single Image

Mandikal, Priyanka; K, L, Navaneet; Agarwal, Mayank; Babu, R. Venkatesh

doi:10.48550/arxiv.1807.07796

Cited by 21 publications

(44 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 1 shows the quantitative comparison between 3D-R2N2 [4], PSGN [6], 3D-LMNet [18] and our proposed method. 3D-R2N2 takes as an input one or more images of an object taken from different viewpoints.…”

Section: Methodsmentioning

confidence: 99%

Inferring Point Clouds from Single Monocular Images by Depth Intermediation

Zeng,

Karaoglu,

Gevers

2018

Preprint

View full text Add to dashboard Cite

In this paper, we propose a framework for generating 3D point cloud of an object from a single-view RGB image. Most previous work predict the 3D point coordinates from single RGB images directly. We decompose this problem into depth estimation from single images and point completion from partial point clouds.Our method sequentially predicts the depth maps and then infers the complete 3D object point clouds based on the predicted partial point clouds. We explicitly impose the camera model geometrical constraint in our pipeline and enforce the alignment of the generated point clouds and estimated depth maps.Experimental results for the single image 3D object reconstruction task show that the proposed method outperforms state-of-the-art methods. Both the qualitative and quantitative results demonstrate the generality and suitability of our method. Camera Model RGB ImagePoint CloudPrevious Methods

show abstract

Section: Methodsmentioning

confidence: 99%

Inferring Point Clouds from Single Monocular Images by Depth Intermediation

Zeng,

Karaoglu,

Gevers

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…To relieve such a problem, recently, some works [42,45,62,7] aim to estimate the truncated signed distance fields to preserve more details. Both voxel-based and point-based methods [13,41,20] require implicit surface reconstruction methods to generate the final triangle mesh. The above methods require ground-truth 3D models, which are difficult to obtain for real scenes.…”

Section: Related Workmentioning

confidence: 99%

Look, Cast and Mold: Learning 3D Shape Manifold from Single-view Synthetic Data

Feng¹,

Luo²,

Luo³

et al. 2021

Preprint

View full text Add to dashboard Cite

Inferring the stereo structure of objects in the real world is a challenging yet practical task. To equip deep models with this ability usually requires abundant 3D supervision which is hard to acquire. It is promising that we can simply benefit from synthetic data, where pairwise groundtruth is easy to access. Nevertheless, the domain gaps are nontrivial considering the variant texture, shape and context. To overcome these difficulties, we propose a Visio-Perceptual Adaptive Network for single-view 3D reconstruction, dubbed VPAN. To generalize the model towards a real scenario, we propose to fulfill several aspects: (1) Look: visually incorporate spatial structure from the single view to enhance the expressiveness of representation; (2) Cast: perceptually align the 2D image features to the 3D shape priors with cross-modal semantic contrastive mapping; (3) Mold: reconstruct stereo-shape of target by transforming embeddings into the desired manifold. Extensive experiments on several benchmarks demonstrate the effectiveness and robustness of the proposed method in learning the 3D shape manifold from synthetic data via a singleview. The proposed method outperforms state-of-the-arts on Pix3D dataset with IoU 0.292 and CD 0.108, and reaches IoU 0.329 and CD 0.104 on Pascal 3D+.

show abstract

“…There has been a lot of research on single image reconstruction task. Recent works involve 3D representation learning, including points [6,13,18], voxels [5,30], meshes [8,26,27,7] and primitives [21,25,29]. The representation can also be learned without the underlying ground truth 3D shapes [11,16,14,33,9,13].…”

Section: Related Workmentioning

confidence: 99%

Neural Implicit 3D Shapes from Single Images with Spatial Patterns

Zhuang¹,

Liu²,

Chen³

2021

Preprint

View full text Add to dashboard Cite

3D shape reconstruction from a single image has been a long-standing problem in computer vision. The problem is ill-posed and highly challenging due to the information loss and occlusion that occurred during the imagery capture. In contrast to previous methods that learn holistic shape priors, we propose a method to learn spatial pattern priors for inferring the invisible regions of the underlying shape, wherein each 3D sample in the implicit shape representation is associated with a set of points generated by hand-crafted 3D mappings, along with their local image features. The proposed spatial pattern is significantly more informative and has distinctive descriptions on both visible and occluded locations. Most importantly, the key to our work is the ubiquitousness of the spatial patterns across shapes, which enables reasoning invisible parts of the underlying objects and thus greatly mitigates the occlusion issue. We devise a neural network that integrates spatial pattern representations and demonstrate the superiority of the proposed method on widely used metrics. Pre-trained models, codes, and data is available at https://github.com/yixin26/SVR-SP.Preprint. Under review.

show abstract

3D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction from a Single Image

Cited by 21 publications

References 0 publications

Inferring Point Clouds from Single Monocular Images by Depth Intermediation

Inferring Point Clouds from Single Monocular Images by Depth Intermediation

Look, Cast and Mold: Learning 3D Shape Manifold from Single-view Synthetic Data

Neural Implicit 3D Shapes from Single Images with Spatial Patterns

Contact Info

Product

Resources

About