Dense 3D Point Cloud Reconstruction Using a Deep Pyramid Network

Mandikal, Priyanka; Radhakrishnan, Venkatesh Babu

doi:10.1109/wacv.2019.00117

Cited by 111 publications

(70 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Point set representation treats a point cloud as a matrix of size N × 3 [21], [22], [68], [71], [73], [77]. • One or multiple 3-channel grids of size H × W × 3 [68], [69], [78].…”

Section: Representationsmentioning

confidence: 99%

See 1 more Smart Citation

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Han

Laga

Bennamoun

2021

IEEE Trans. Pattern Anal. Mach. Intell.

320

150

View full text Add to dashboard Cite

3D reconstruction is a longstanding ill-posed problem, which has been explored for decades by the computer vision, computer graphics, and machine learning communities. Since 2015, image-based 3D reconstruction using convolutional neural networks (CNN) has attracted increasing interest and demonstrated an impressive performance. Given this new era of rapid evolution, this article provides a comprehensive survey of the recent developments in this field. We focus on the works which use deep learning techniques to estimate the 3D shape of generic objects either from a single or multiple RGB images. We organize the literature based on the shape representations, the network architectures, and the training mechanisms they use. While this survey is intended for methods which reconstruct generic objects, we also review some of the recent works which focus on specific object classes such as human body shapes and faces. We provide an analysis and comparison of the performance of some key papers, summarize some of the open problems in this field, and discuss promising directions for future research.

show abstract

“…• Point set representation treats a point cloud as a matrix of size N × 3 [21], [22], [68], [71], [73], [77]. • One or multiple 3-channel grids of size H × W × 3 [68], [69], [78].…”

Section: Representationsmentioning

confidence: 99%

“…Point set representations ( Fig. 3-(c)) use fully connected layers [21], [68], [70], [73], [77] since point clouds are unordered. The main advantage of fully-connected layers is that they capture the global information.…”

Section: Network Architecturesmentioning

confidence: 99%

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Han

Laga

Bennamoun

2021

IEEE Trans. Pattern Anal. Mach. Intell.

320

150

View full text Add to dashboard Cite

show abstract

“…Mandikal et al [30] presented a latent-embedding matching method for 3D reconstruction called 3D-LMNet which handled the problem of ambiguous ground truths and evaluated the effectiveness of the method by generating multiple plausible 3D objects. He also developed Dense 3D Point Cloud Reconstruction (DensePCR) [31], a deep pyramid network, to hierarchically predict point cloud from the previous low-resolution point cloud.…”

Section: Related Workmentioning

confidence: 99%

Single Image Shape-from-Silhouettes

Wang

2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Recovering a 3D shape representation from one single image input has been attempted in recent years. Most of the works obtain 3D models from multiple images at different perspectives or ground truth CAD models. However, multiple images from different perspectives or 3D CAD models are not always available in real applications. In this work, we present a novel shape-from-silhouette method based on just a single image, which is an end-to-end learning framework relying on view synthesis and shape-from-silhouette methodology to reconstruct a 3D shape. The reconstructed 3D mesh can approach the real shape of target objects by constraining the silhouettes from both horizontal and vertical directions, especially for those objects with occlusions. Our proposed method achieves stateof-the-art performance on the ShapeNet dataset compared with other recent approaches targeting 3D reconstruction from a single image. Without requiring labor-intensive and time-consuming human annotations, the work has a broad potential to be applied in real-world applications. CCS CONCEPTS • Information systems → Multimedia content creation.

show abstract

“…Besides, Li et al [ 29 ] also propose the PC-GAN, which is directly extended from the GAN to the generation of 3D point clouds. Mandikal et al [ 30 ] propose the Dense-PCR network, which is a deep pyramid network used for the point cloud reconstruction. A low-resolution point cloud algorithm is proposed, to achieve grid deformation by aggregating local and global point features, to increase the resolution of the grid hierarchically.…”

Section: Related Workmentioning

confidence: 99%

3DMGNet: 3D Model Generation Network Based on Multi-Modal Data Constraints and Multi-Level Feature Fusion

Wang

Xue

et al. 2020

Sensors

View full text Add to dashboard Cite

Due to the limitation of less information in a single image, it is very difficult to generate a high-precision 3D model based on the image. There are some problems in the generation of 3D voxel models, e.g., the information loss at the upper level of a network. To solve these problems, we design a 3D model generation network based on multi-modal data constraints and multi-level feature fusion, named as 3DMGNet. Moreover, 3DMGNet is trained by self-supervised method to achieve 3D voxel model generation from an image. The image feature extraction network (2DNet) and 3D feature extraction network (3D auxiliary network) are used to extract the features of the image and 3D voxel model. Then, feature fusion is used to integrate the low-level features into the high-level features in the 3D auxiliary network. To extract more effective features, each layer of the feature map in feature extraction network is processed by an attention network. Finally, the extracted features generate 3D models by a 3D deconvolution network. The feature extraction of 3D model and the generation of voxelization play an auxiliary role in the training of the whole network for the 3D model generation based on an image. Additionally, a multi-view contour constraint method is proposed, to enhance the effect of the 3D model generation. In the experiment, the ShapeNet dataset is adapted to prove the effect of the 3DMGNet, which verifies the robust performance of the proposed method.

show abstract

Dense 3D Point Cloud Reconstruction Using a Deep Pyramid Network

Cited by 111 publications

References 24 publications

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era

Single Image Shape-from-Silhouettes

3DMGNet: 3D Model Generation Network Based on Multi-Modal Data Constraints and Multi-Level Feature Fusion

Contact Info

Product

Resources

About