PU-Net: Point Cloud Upsampling Network

Abstract

Learning and analyzing 3D point clouds with deep networks is challenging due to the sparseness and irregularity of the data. In this paper, we present a data-driven point cloud upsampling technique. The key idea is to learn multilevel features per point and expand the point set via a multibranch convolution unit implicitly in feature space. The expanded feature is then split to a multitude of features, which are then reconstructed to an upsampled point set. Our network is applied at a patch-level, with a joint loss function that encourages the upsampled points to remain on the underlying surface with a uniform distribution. We conduct various experiments using synthesis and scan data to evaluate our method and demonstrate its superiority over some baseline methods and an optimization-based method. Results show that our upsampled points have better uniformity and are located closer to the underlying surfaces.

PU-GAN: A Point Cloud Upsampling Adversarial Network

“…Different from point completion [41,10] which generates the entire object from partial input, point cloud upsampling tends to improve the point distribution uniformity within local patches. Based on the Point-Net++ architecture, Yu et al [40] introduced a deep neural network PU-Net to upsample point sets. PU-Net works on patches and expands a point set by mixing-and-blending point features in the feature space.…”

“…Thus, we formulate a uniform loss to evaluate Q from the generator, aiming to improve the generative ability of the generator. To evaluate a point set's uniformity, the NUC metric in PU-Net [40] crops equal-sized disks on the object surface and counts the number of points in the disks. So, the metric neglects the local clutter of points.…”

“…We qualitatively and quantitatively compared PU-GAN with three state-of-the-art point set upsampling methods: EAR [15], PU-Net [40], and MPU [38]. For EAR, we employed the released demo code and generated the best results by exhaustively fine-tuning every associated parameter.…”

“…These goals are very challenging to achieve, given such an imperfect input, since, apart from 1 https://liruihui.github.io/publication/PU-GAN/ Figure 1. Upsampling (b) a point set cropped from (a) the realscanned KITTI dataset [8] by using (c) PU-Net [40] (CVPR 2018), (d) MPU [38] (CVPR 2019), and (e) our PU-GAN. Note the distribution non-uniformity and sparsity in the input.…”

“…Recently, deep neural networks brought the promise of data-driven approaches to the problem. Network architectures, including PU-Net [40], EC-Net [39], and very recently, MPU [38], have demonstrated the advantage of upsampling point clouds through learning. However, these networks may not produce plausible results from low-quality inputs that are particularly sparse and non-uniform; in Figure 1 (c)-(e), we show a typical example that demonstrates the advantage of our method, which unlike the other networks, it combines completion and uniformity together with upsampling.…”