3D-PSRNet: Part Segmented 3D Point Cloud Reconstruction from a Single Image

Mandikal, Priyanka; Navaneet, K L; Babu, R. Venkatesh

doi:10.1007/978-3-030-11015-4_50

Cited by 98 publications

(99 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…• Point set representation treats a point cloud as a matrix of size N × 3 [21], [22], [68], [71], [73], [77]. • One or multiple 3-channel grids of size H × W × 3 [68], [69], [78].…”

Section: Representationsmentioning

confidence: 99%

See 1 more Smart Citation

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Han

Laga

Bennamoun

2021

IEEE Trans. Pattern Anal. Mach. Intell.

320

150

View full text Add to dashboard Cite

3D reconstruction is a longstanding ill-posed problem, which has been explored for decades by the computer vision, computer graphics, and machine learning communities. Since 2015, image-based 3D reconstruction using convolutional neural networks (CNN) has attracted increasing interest and demonstrated an impressive performance. Given this new era of rapid evolution, this article provides a comprehensive survey of the recent developments in this field. We focus on the works which use deep learning techniques to estimate the 3D shape of generic objects either from a single or multiple RGB images. We organize the literature based on the shape representations, the network architectures, and the training mechanisms they use. While this survey is intended for methods which reconstruct generic objects, we also review some of the recent works which focus on specific object classes such as human body shapes and faces. We provide an analysis and comparison of the performance of some key papers, summarize some of the open problems in this field, and discuss promising directions for future research.

show abstract

“…• Point set representation treats a point cloud as a matrix of size N × 3 [21], [22], [68], [71], [73], [77]. • One or multiple 3-channel grids of size H × W × 3 [68], [69], [78].…”

Section: Representationsmentioning

confidence: 99%

“…Point set representations ( Fig. 3-(c)) use fully connected layers [21], [68], [70], [73], [77] since point clouds are unordered. The main advantage of fully-connected layers is that they capture the global information.…”

Section: Network Architecturesmentioning

confidence: 99%

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Han

Laga

Bennamoun

2021

IEEE Trans. Pattern Anal. Mach. Intell.

320

150

View full text Add to dashboard Cite

show abstract

“…There has been much 3D model generation work based on the point cloud. For example, Mandikal et al [ 26 ] propose 3DLMNet, in which a latent space is learnt by an auto-encoder network, and the image is encoded into the space. With the constraints of difference between the feature of the image and the feature of the point cloud, a space vector that can generate an image of a 3D model is obtained.…”

Section: Related Workmentioning

confidence: 99%

3DMGNet: 3D Model Generation Network Based on Multi-Modal Data Constraints and Multi-Level Feature Fusion

Wang

Xue

et al. 2020

Sensors

View full text Add to dashboard Cite

Due to the limitation of less information in a single image, it is very difficult to generate a high-precision 3D model based on the image. There are some problems in the generation of 3D voxel models, e.g., the information loss at the upper level of a network. To solve these problems, we design a 3D model generation network based on multi-modal data constraints and multi-level feature fusion, named as 3DMGNet. Moreover, 3DMGNet is trained by self-supervised method to achieve 3D voxel model generation from an image. The image feature extraction network (2DNet) and 3D feature extraction network (3D auxiliary network) are used to extract the features of the image and 3D voxel model. Then, feature fusion is used to integrate the low-level features into the high-level features in the 3D auxiliary network. To extract more effective features, each layer of the feature map in feature extraction network is processed by an attention network. Finally, the extracted features generate 3D models by a 3D deconvolution network. The feature extraction of 3D model and the generation of voxelization play an auxiliary role in the training of the whole network for the 3D model generation based on an image. Additionally, a multi-view contour constraint method is proposed, to enhance the effect of the 3D model generation. In the experiment, the ShapeNet dataset is adapted to prove the effect of the 3DMGNet, which verifies the robust performance of the proposed method.

show abstract

“…Regressing 3D coordinates from 2D observations is a well-studied problem in computer vision [16]. While traditional approaches generate 3D point clouds from multi-view RGB images [51,54], recent works predict unstructured 3D point clouds from a single RGB image using deep learning [12,35,38]. Complementary to these works, Point-Net [42] and successors [29,43] showed that even such unstructured 3D point clouds can be used to address various 3D vision tasks with deep learning.…”

Section: Location Fieldsmentioning

confidence: 99%

Location Field Descriptors: Single Image 3D Model Retrieval in the Wild

Grabner

Roth

Lepetit

2019

2019 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

We present Location Field Descriptors, a novel approach for single image 3D model retrieval in the wild. In contrast to previous methods that directly map 3D models and RGB images to an embedding space, we establish a common low-level representation in the form of location fields from which we compute pose invariant 3D shape descriptors. Location fields encode correspondences between 2D pixels and 3D surface coordinates and, thus, explicitly capture 3D shape and 3D pose information without appearance variations which are irrelevant for the task. This early fusion of 3D models and RGB images results in three main advantages: First, the bottleneck location field prediction acts as a regularizer during training. Second, major parts of the system benefit from training on a virtually infinite amount of synthetic data. Finally, the predicted location fields are visually interpretable and unblackbox the system. We evaluate our proposed approach on three challenging real-world datasets (Pix3D, Comp, and Stanford) with different object categories and significantly outperform the state-of-the-art by up to 20% absolute in multiple 3D retrieval metrics.

show abstract

3D-PSRNet: Part Segmented 3D Point Cloud Reconstruction from a Single Image

Cited by 98 publications

References 20 publications

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

3DMGNet: 3D Model Generation Network Based on Multi-Modal Data Constraints and Multi-Level Feature Fusion

Location Field Descriptors: Single Image 3D Model Retrieval in the Wild

Contact Info

Product

Resources

About