High-Fidelity Point Cloud Completion with Low-Resolution Recovery and Noise-Aware Upsampling

Abstract

Completing an unordered partial point cloud is a challenging task. Existing approaches that rely on decoding a latent feature to recover the complete shape, often lead to the completed point cloud being oversmoothing, losing details, and noisy. Instead of decoding a whole shape, we propose to decode and refine a lowresolution (low-res) point cloud first, and then performs a patch-wise noise-aware upsampling rather than interpolating the whole sparse point cloud at once, which tends to lose details. Regarding the possibility of lacking details of the initially decoded low-res point cloud, we propose an iterative refinement to recover the geometric details and a symmetrization process to preserve the trustworthy information from the input partial point cloud. After obtaining a sparse and complete point cloud, we propose a patch-wise upsampling strategy. Patch-based upsampling allows to better recover fine details unlike decoding a whole shape, however, the existing upsampling methods are not applicable to completion task due to the data discrepancy (i.e., input sparse data here is not from ground-truth). Therefore, we propose a patch extraction approach to generate training patch pairs between the sparse and ground-truth point clouds, and an outlier removal step to suppress the noisy points from the sparse point cloud. Together with the low-res recovery, our whole method is able to achieve high-fidelity point cloud completion. Comprehensive evaluations are provided to demonstrate the effectiveness of the proposed method and its individual components.

Comprehensive Review of Deep Learning-Based 3D Point Cloud Completion Processing and Analysis

“…To tackle the challenging dense 3D point cloud completion problem, Li et al [41] proposed a framework to perform endto-end low-resolution recovery first, followed by a patch-wise noise aware upsampling. This method achieves a high-fidelity dense point cloud completion through decoding a complete but sparse shape, iterative refinement, preserving trustworthy information by symmetrization, and patch-wise up-sampling.…”

Point Cloud Completion: A Survey

“…Any point outside the boundary can be removed by Farthest Point Sampling (FPS) and thus a cleaner output is made. Li et al [117] use an outlier removal process to suppress outlier points from sparse point clouds during the patch-training process.…”

“…Hyperpocket [34] splits the point cloud processing into two disjoint data streams of partial input and "pockets": empty spaces left by the missing part of the objects. Li et al [117] propose to decode a lowresolution point cloud first and perform path-wise noise-aware up-sampling and recover the details patch by patch. More works that use parallel, dual-path networks include [180] [181].…”