Sign-Agnostic Implicit Learning of Surface Self-Similarities for Shape Modeling and Reconstruction from Raw Point Clouds

Zhao, Wenbin; Lei, Jiabao; Wen, Yuxin; Zhang, Jianguo; Jia, Kui

doi:10.1109/cvpr46437.2021.01012

Cited by 25 publications

(13 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motivated from deep image prior [63], Deep Geometric Prior (DGP) [64] verifies the efficacy of deep networks as a prior for geometric surface modeling, even when the networks are not trained. Latter on, Point2Mesh [65] and SAIL-S3 [66] extend the global modeling adopted in [64] as local ones, where the former constructs its local, implicit functions as a weight-shared MeshCNN [67] and the later method constructs them as a weightshared Multi-Layer Perceptron (MLP). Deep Manifold Prior [68] delivers mathematical analyses on such modeling properties for MLP as well as convolutional networks.…”

Section: Modeling Priorsmentioning

confidence: 99%

Surface Reconstruction from Point Clouds: A Survey and a Benchmark

Huang¹,

Wen²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Reconstruction of a continuous surface of two-dimensional manifold from its raw, discrete point cloud observation is a long-standing problem in computer vision and graphics research. The problem is technically ill-posed, and becomes more difficult considering that various sensing imperfections would appear in the point clouds obtained by practical depth scanning. In literature, a rich set of methods has been proposed, and reviews of existing methods are also provided. However, existing reviews are short of thorough investigations on a common benchmark. The present paper aims to review and benchmark existing methods in the new era of deep learning surface reconstruction. To this end, we contribute a large-scale benchmarking dataset consisting of both synthetic and real-scanned data; the benchmark includes object-and scene-level surfaces and takes into account various sensing imperfections that are commonly encountered in practical depth scanning. We conduct thorough empirical studies by comparing existing methods on the constructed benchmark, and pay special attention on robustness of existing methods against various scanning imperfections; we also study how different methods generalize in terms of reconstructing complex surface shapes. Our studies help identify the best conditions under which different methods work, and suggest some empirical findings. For example, while deep learning methods are increasingly popular in the research community, our systematic studies suggest that, surprisingly, a few classical methods perform even better in terms of both robustness and generalization; our studies also suggest that the practical challenges of misalignment of point sets from multi-view scanning, missing of surface points, and point outliers remain unsolved by all the existing surface reconstruction methods. We expect that the benchmark and our studies would be valuable both for practitioners and as a guidance for new innovations in future research. We make the benchmark publicly accessible at https://Gorilla-Lab-SCUT.github.io/SurfaceReconstructionBenchmark.

show abstract

Section: Modeling Priorsmentioning

confidence: 99%

Surface Reconstruction from Point Clouds: A Survey and a Benchmark

Huang¹,

Wen²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A solution is to split the input points on a regular 3D grid and to optimize one latent vector per voxel [12] (DeepLS), possibly from overlapping input patches. Patch splitting can also be irregular and optimization-driven to favor selfsimilarities, with a global post-optimization to flip inconsistent local signs [129] (SAIL-S3). But whether these methods optimize only the latent vectors or a whole network as well, for patch decoding, they make surface reconstruction significantly slower, leading to reduced test sets.…”

Section: Related Work 21 3d Representationsmentioning

confidence: 99%

“…Departing from occupancy or distance fields, ShapeGF [11] models a shape by learning the gradient field of its log-density, then samples points on high likelihood regions of the shape and meshes them. Other work also study the decomposition of shapes and implicit surfaces into parts [35,34,84,28,48,109], possibly overfitting networks to generate or render a single object or scene [117,101,65,129,103,73,126].…”

Section: Related Work 21 3d Representationsmentioning

confidence: 99%

POCO: Point Convolution for Surface Reconstruction

Boulch¹,

Marlet²

2022

Preprint

View full text Add to dashboard Cite

Implicit neural networks have been successfully used for surface reconstruction from point clouds. However, many of them face scalability issues as they encode the isosurface function of a whole object or scene into a single latent vector. To overcome this limitation, a few approaches infer latent vectors on a coarse regular 3D grid or on 3D patches, and interpolate them to answer occupancy queries. In doing so, they loose the direct connection with the input points sampled on the surface of objects, and they attach information uniformly in space rather than where it matters the most, i.e., near the surface. Besides, relying on fixed patch sizes may require discretization tuning. To address these issues, we propose to use point cloud convolutions and compute latent vectors at each input point. We then perform a learning-based interpolation on nearest neighbors using inferred weights. Experiments on both object and scene datasets show that our approach significantly outperforms other methods on most classical metrics, producing finer details and better reconstructing thinner volumes. The code is available at https://github.com/ valeoai/POCO.

show abstract

“…Iso-points [83] tried to impose geometry-aware sampling and regularization in the learning. Moreover, implicit functions can also be learned from point clouds with additional constraints, such as geometric regularization [19], sign agnostic learning with a specially designed loss function [1], sign agnostic learning with local surface self-similarities and post sign processing [71,85], constraints on gradients [2] or a divergence penalty [4].…”

Section: Related Workmentioning

confidence: 99%

Surface Reconstruction from Point Clouds by Learning Predictive Context Priors

Baorui¹,

Liu²,

Zwicker³

et al. 2022

Preprint

View full text Add to dashboard Cite

Surface reconstruction from point clouds is vital for 3D computer vision. State-of-the-art methods leverage large datasets to first learn local context priors that are represented as neural network-based signed distance functions (SDFs) with some parameters encoding the local contexts. To reconstruct a surface at a specific query location at inference time, these methods then match the local reconstruction target by searching for the best match in the local prior space (by optimizing the parameters encoding the local context) at the given query location. However, this requires the local context prior to generalize to a wide variety of unseen target regions, which is hard to achieve.

show abstract

Sign-Agnostic Implicit Learning of Surface Self-Similarities for Shape Modeling and Reconstruction from Raw Point Clouds

Cited by 25 publications

References 23 publications

Surface Reconstruction from Point Clouds: A Survey and a Benchmark

Surface Reconstruction from Point Clouds: A Survey and a Benchmark

POCO: Point Convolution for Surface Reconstruction

Surface Reconstruction from Point Clouds by Learning Predictive Context Priors

Contact Info

Product

Resources

About