Neural Splines: Fitting 3D Surfaces with Infinitely-Wide Neural Networks

Williams, Francis; Trager, Matthew; Bruna, Joan; Zorin, Denis

doi:10.1109/cvpr46437.2021.00982

Cited by 42 publications

(23 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, although it can cope with a substantial variation of point density, it cannot complete shapes when large parts are missing. Apart from a few methods like [26,27,25,98] 8.58 0.936 0.9787 Neural Splines [118] 5.99 0.982 0.9958 LIG [49] 8.69 0.975 0.9773 POCO (ours) 5.27 0.992 0.9987 1000 SPR10 [51] 7.29 0.967 0.9957 LIG [49] 8.40 0.978 0.9750 POCO (ours) 5.34 0.993 0.9987 Oracle 5.02 0.995 0.9998 [118] uses a grid size of 1024, 10k Nyström samples, 8×8×8 chunks. Numbers differ from [49] as we had to regenerate the unavailable watertight meshes.…”

Section: Discussion and Limitationsmentioning

confidence: 99%

“…Traditional 3D reconstruction approaches [6] generally express the target surface as the solution to an optimization problem under some prior constraints. Possibly leveraging visibility or normal information, they are generally scalable to large scenes and offer a substantial robustness to noise and outliers [56,77,111,51,131,102,118,88]. Although some try to cope with density variation [46,47,10], a common limitation of these approaches is their inability to properly complete parts of the scene that are less densely sam- pled or that are missing (typically due to occlusions).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

POCO: Point Convolution for Surface Reconstruction

Boulch¹,

Marlet²

2022

Preprint

View full text Add to dashboard Cite

Implicit neural networks have been successfully used for surface reconstruction from point clouds. However, many of them face scalability issues as they encode the isosurface function of a whole object or scene into a single latent vector. To overcome this limitation, a few approaches infer latent vectors on a coarse regular 3D grid or on 3D patches, and interpolate them to answer occupancy queries. In doing so, they loose the direct connection with the input points sampled on the surface of objects, and they attach information uniformly in space rather than where it matters the most, i.e., near the surface. Besides, relying on fixed patch sizes may require discretization tuning. To address these issues, we propose to use point cloud convolutions and compute latent vectors at each input point. We then perform a learning-based interpolation on nearest neighbors using inferred weights. Experiments on both object and scene datasets show that our approach significantly outperforms other methods on most classical metrics, producing finer details and better reconstructing thinner volumes. The code is available at https://github.com/ valeoai/POCO.

show abstract

Section: Discussion and Limitationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

POCO: Point Convolution for Surface Reconstruction

Boulch¹,

Marlet²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Sign Agnostic Learning (SAL) [72] and its variants [73], [74] study reconstructing a surface from an un-oriented point cloud via a specially initialized neural network, and Davies et al [75] adopt the same strategy for surface reconstruction from an oriented point cloud. Neural Splines [76] performs reconstruction based on random feature kernels arising from infinitely-wide shallow ReLU networks and shows that such solutions bias toward reconstruction of smooth surface.…”

Section: Modeling Priorsmentioning

confidence: 99%

Surface Reconstruction from Point Clouds: A Survey and a Benchmark

Huang¹,

Wen²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Reconstruction of a continuous surface of two-dimensional manifold from its raw, discrete point cloud observation is a long-standing problem in computer vision and graphics research. The problem is technically ill-posed, and becomes more difficult considering that various sensing imperfections would appear in the point clouds obtained by practical depth scanning. In literature, a rich set of methods has been proposed, and reviews of existing methods are also provided. However, existing reviews are short of thorough investigations on a common benchmark. The present paper aims to review and benchmark existing methods in the new era of deep learning surface reconstruction. To this end, we contribute a large-scale benchmarking dataset consisting of both synthetic and real-scanned data; the benchmark includes object-and scene-level surfaces and takes into account various sensing imperfections that are commonly encountered in practical depth scanning. We conduct thorough empirical studies by comparing existing methods on the constructed benchmark, and pay special attention on robustness of existing methods against various scanning imperfections; we also study how different methods generalize in terms of reconstructing complex surface shapes. Our studies help identify the best conditions under which different methods work, and suggest some empirical findings. For example, while deep learning methods are increasingly popular in the research community, our systematic studies suggest that, surprisingly, a few classical methods perform even better in terms of both robustness and generalization; our studies also suggest that the practical challenges of misalignment of point sets from multi-view scanning, missing of surface points, and point outliers remain unsolved by all the existing surface reconstruction methods. We expect that the benchmark and our studies would be valuable both for practitioners and as a guidance for new innovations in future research. We make the benchmark publicly accessible at https://Gorilla-Lab-SCUT.github.io/SurfaceReconstructionBenchmark.

show abstract

“…Compared to volumetric-based representations, the coordinate-based MLPs are memory efficient and can preserve the continuity of the resulting 3D volumetric field. Therefore, they have been applied for reconstructing the continuous distance field of 3D shapes from point clouds via unsupervised learning Lipman 2020, 2021;Gropp et al 2020] or supervised learning [Williams et al 2022[Williams et al , 2021. However, large amounts of computations are required to reconstruct high-resolution fields with the coordinate-based MLPs.…”

Section: Related Workmentioning

confidence: 99%

Dual Octree Graph Networks for Learning Adaptive Volumetric Shape Representations

Wang,

Liu,

Tong

2022

Preprint

View full text Add to dashboard Cite

Neural Splines: Fitting 3D Surfaces with Infinitely-Wide Neural Networks

Cited by 42 publications

References 22 publications

POCO: Point Convolution for Surface Reconstruction

POCO: Point Convolution for Surface Reconstruction

Surface Reconstruction from Point Clouds: A Survey and a Benchmark

Dual Octree Graph Networks for Learning Adaptive Volumetric Shape Representations

Contact Info

Product

Resources

About