From Image Collections to Point Clouds With Self-Supervised Shape and Pose Networks

Navaneet, K L; Mathew, Ansu; Kashyap, Shashank; Hung, Wei-Chih; Jampani, Varun; Babu, R. Venkatesh

doi:10.1109/cvpr42600.2020.00121

Cited by 27 publications

(7 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ill-posed problem of 3D reconstruction from a single image is addressed by their proposed two-pronged method. Other methods about 3D point clouds reconstruction from a single image can also be found in [27][28][29][30].…”

Section: Related Workmentioning

confidence: 99%

PDE-Based 3D Surface Reconstruction from Multi-View 2D Images

Zhu

Iglesias

et al. 2022

Mathematics

View full text Add to dashboard Cite

Partial differential equation (PDE) based surfaces own a lot of advantages, compared to other types of 3D representation. For instance, fewer variables are required to represent the same 3D shape; the position, tangent, and even curvature continuity between PDE surface patches can be naturally maintained when certain conditions are satisfied, and the physics-based nature is also kept. Although some works applied implicit PDEs to 3D surface reconstruction from images, there is little work on exploiting the explicit solutions of PDE to this topic, which is more efficient and accurate. In this paper, we propose a new method to apply the explicit solutions of a fourth-order partial differential equation to surface reconstruction from multi-view images. The method includes two stages: point clouds data are extracted from multi-view images in the first stage, which is followed by PDE-based surface reconstruction from the obtained point clouds data. Our computational experiments show that the reconstructed PDE surfaces exhibit good quality and can recover the ground truth with high accuracy. A comparison between various solutions with different complexity to the fourth-order PDE is also made to demonstrate the power and flexibility of our proposed explicit PDE for surface reconstruction from images.

show abstract

Section: Related Workmentioning

confidence: 99%

PDE-Based 3D Surface Reconstruction from Multi-View 2D Images

Zhu

Iglesias

et al. 2022

Mathematics

View full text Add to dashboard Cite

show abstract

“…Navaneet et al. [26] adopted an unsupervised method to predict point clouds for 2D objects. TDPNet [27] focussed on a wealth of information available on shape in 3D data by integrating image features with 3D prototype features.…”

Section: Related Workmentioning

confidence: 99%

3D‐FEGNet: A feature enhanced point cloud generation network from a single image

Wang

Hui

Wang

et al. 2022

IET Computer Vision

View full text Add to dashboard Cite

Deep learning‐based single view 3D reconstruction is a hot topic in computer vision. However, predicting a more realistic 3D point cloud from a single image is an ill‐posed problem. In recent years, most of the 3D point cloud prediction methods based on single view are straight‐through structure, which will cause the loss of part of feature information and the loss of part of detail information of the resulting point clouds, which will lead to the unsatisfactory visual effect of reconstructed point clouds. In this paper, a Feature‐Enhanced 3D point clouds generation Network (3D‐FENet) from a single image is proposed. In order to enhance the feature information of RGB image, edge extraction module is adopted. In the process of point cloud generation, a point cloud pyramid is designed, which combines low resolution point cloud with high resolution point cloud to enhance the local details of the generated point clouds. In the fine‐tuning stage, the differential projection module is used to fine‐tune the whole network by 2D projection of reconstructed point clouds. Experimental results show that the performance of the authors’ proposed method is better than the state‐of‐the‐art studies.

show abstract

“…AtlasNet (Groueix et al 2018) trained a set of manifold decoders that are applicable to both point cloud self-recontruction and image-to-point-cloud synthesization. A contemporary method, so-called SSPNet (Navaneet et al 2020), generates point cloud from one image by enforcing geometric and pose cycle consistency. Nevertheless, it has a strong assumption that each image has its corresponding sihouette (image mask).…”

Section: Single-view 3d Reconstructionmentioning

confidence: 99%

Single View Point Cloud Generation via Unified 3D Prototype

Lin¹,

Wang²,

Li³

et al. 2021

AAAI

View full text Add to dashboard Cite

As 3D point clouds become the representation of choice for multiple vision and graphics applications, such as autonomous driving, robotics, etc., the generation of them by deep neural networks has attracted increasing attention in the research community. Despite the recent success of deep learning models in classification and segmentation, synthesizing point clouds remains challenging, especially from a single image. State-of-the-art (SOTA) approaches can generate a point cloud from a hidden vector, however, they treat 2D and 3D features equally and disregard the rich shape information within the 3D data. In this paper, we address this problem by integrating image features with 3D prototype features. Specifically, we propose to learn a set of 3D prototype features from a real point cloud dataset and dynamically adjust them through the training. These prototypes are then integrated with incoming image features to guide the point cloud generation process. Experimental results show that our proposed method outperforms SOTA methods on single image based 3D reconstruction tasks.

show abstract

From Image Collections to Point Clouds With Self-Supervised Shape and Pose Networks

Cited by 27 publications

References 8 publications

PDE-Based 3D Surface Reconstruction from Multi-View 2D Images

PDE-Based 3D Surface Reconstruction from Multi-View 2D Images

3D‐FEGNet: A feature enhanced point cloud generation network from a single image

Single View Point Cloud Generation via Unified 3D Prototype

Contact Info

Product

Resources

About