3D reconstruction using deep learning: a survey

Jin, Yiwei; Jiang, Diqiong; Cai, Meifeng

doi:10.4310/cis.2020.v20.n4.a1

Cited by 35 publications

(16 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternatively, as a mesh can be represented as a graph, CNNs can be extended to graph CNNs and novel mesh pooling [54] in the spatial domain [55], [56], or spectral domain [57], [58], [59], [60] for mesh-based deep learning. For the comprehensive understanding on different 3D representation for deep learning, please refer to these surveys [61], [62], [63], [64]. In the spatial domain, works [65], [66], [67], [68] apply variational autoencoders on 3D meshes for various applications such as reconstruction, interpolation, completion and embedding.…”

Section: Related Workmentioning

confidence: 99%

Multiscale Mesh Deformation Component Analysis With Attention-Based Autoencoders

Yang

Gao

Tan

et al. 2023

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Deformation component analysis is a fundamental problem in geometry processing and shape understanding. Existing approaches mainly extract deformation components in local regions at a similar scale while deformations of real-world objects are usually distributed in a multi-scale manner. In this paper, we propose a novel method to exact multiscale deformation components automatically with a stacked attention-based autoencoder. The attention mechanism is designed to learn to softly weight multi-scale deformation components in active deformation regions, and the stacked attention-based autoencoder is learned to represent the deformation components at different scales. Quantitative and qualitative evaluations show that our method outperforms state-of-the-art methods. Furthermore, with the multiscale deformation components extracted by our method, the user can edit shapes in a coarse-to-fine fashion which facilitates effective modeling of new shapes.

show abstract

Section: Related Workmentioning

confidence: 99%

Multiscale Mesh Deformation Component Analysis With Attention-Based Autoencoders

Yang

Gao

Tan

et al. 2023

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

“…Representations for 3D Shape Learning. Various 3D representations have been extensively studied in 3D deep learning [28]. These surveys [2,62] discuss various shape representations comprehensively.…”

Section: Related Workmentioning

confidence: 99%

OctField: Hierarchical Implicit Functions for 3D Modeling

Tang¹,

Chen²,

Jie

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent advances in localized implicit functions have enabled neural implicit representation to be scalable to large scenes. However, the regular subdivision of 3D space employed by these approaches fails to take into account the sparsity of the surface occupancy and the varying granularities of geometric details. As a result, its memory footprint grows cubically with the input volume, leading to a prohibitive computational cost even at a moderately dense decomposition. In this work, we present a learnable hierarchical implicit representation for 3D surfaces, coded OctField, that allows high-precision encoding of intricate surfaces with low memory and computational budget. The key to our approach is an adaptive decomposition of 3D scenes that only distributes local implicit functions around the surface of interest. We achieve this goal by introducing a hierarchical octree structure to adaptively subdivide the 3D space according to the surface occupancy and the richness of part geometry. As octree is discrete and non-differentiable, we further propose a novel hierarchical network that models the subdivision of octree cells as a probabilistic process and recursively encodes and decodes both octree structure and surface geometry in a differentiable manner. We demonstrate the value of OctField for a range of shape modeling and reconstruction tasks, showing superiority over alternative approaches.

show abstract

“…Analyzing and understanding the 3D world [7,29,27,4,37,20,30,36,40,16,52,23,18] plays an increasingly important role in the rapid development of autonomous driv-ing, robotics, and 3D reconstruction [12,31,9]. Among different 3D data representations, point cloud data has received much attention.…”

Section: Introductionmentioning

confidence: 99%

High-Fidelity Point Cloud Completion with Low-Resolution Recovery and Noise-Aware Upsampling

Li¹,

Wang²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Completing an unordered partial point cloud is a challenging task. Existing approaches that rely on decoding a latent feature to recover the complete shape, often lead to the completed point cloud being oversmoothing, losing details, and noisy. Instead of decoding a whole shape, we propose to decode and refine a lowresolution (low-res) point cloud first, and then performs a patch-wise noise-aware upsampling rather than interpolating the whole sparse point cloud at once, which tends to lose details. Regarding the possibility of lacking details of the initially decoded low-res point cloud, we propose an iterative refinement to recover the geometric details and a symmetrization process to preserve the trustworthy information from the input partial point cloud. After obtaining a sparse and complete point cloud, we propose a patch-wise upsampling strategy. Patch-based upsampling allows to better recover fine details unlike decoding a whole shape, however, the existing upsampling methods are not applicable to completion task due to the data discrepancy (i.e., input sparse data here is not from ground-truth). Therefore, we propose a patch extraction approach to generate training patch pairs between the sparse and ground-truth point clouds, and an outlier removal step to suppress the noisy points from the sparse point cloud. Together with the low-res recovery, our whole method is able to achieve high-fidelity point cloud completion. Comprehensive evaluations are provided to demonstrate the effectiveness of the proposed method and its individual components.

show abstract

3D reconstruction using deep learning: a survey

Cited by 35 publications

References 0 publications

Multiscale Mesh Deformation Component Analysis With Attention-Based Autoencoders

Multiscale Mesh Deformation Component Analysis With Attention-Based Autoencoders

OctField: Hierarchical Implicit Functions for 3D Modeling

High-Fidelity Point Cloud Completion with Low-Resolution Recovery and Noise-Aware Upsampling

Contact Info

Product

Resources

About