Multi-Angle Point Cloud-VAE: Unsupervised Feature Learning for 3D Point Clouds From Multiple Angles by Joint Self-Reconstruction and Half-to-Half Prediction

Han, Zhizhong; Wang, Xiyang; Liu, Yu‐Shen; Zwicker, Matthias

doi:10.1109/iccv.2019.01054

Cited by 137 publications

(78 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3D-GAN [49] 83.3% Latent-GAN [1] 85.7% SO-Net [21] 87.3% MAP-VAE [16] 88.4% Jigsaw * [42] 84.1% FoldingNet * [54] 90.1% Orientation * [36] 90.7% STRL * [20] 90.9% OcCo * [47] 89.7% IAE(ours) 92.1%…”

Section: Methods Modelnet40mentioning

confidence: 99%

Implicit Autoencoder for Point Cloud Self-supervised Representation Learning

Shen¹,

Yang²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Many 3D representations (e.g., point clouds) are discrete samples of the underlying continuous 3D surface. This process inevitably introduces sampling variations on the underlying 3D shapes. In learning 3D representation, the variations should be disregarded while transferable knowledge of the underlying 3D shape should be captured. This becomes a grand challenge in existing representation learning paradigms. This paper studies autoencoding on point clouds. The standard autoencoding paradigm forces the encoder to capture such sampling variations as the decoder has to reconstruct the original point cloud that has sampling variations. We introduce Implicit Autoencoder(IAE), a simple yet effective method that addresses this challenge by replacing the point cloud decoder with an implicit decoder. The implicit decoder outputs a continuous representation that is shared among different point cloud sampling of the same model. Reconstructing under the implicit representation can prioritize that the encoder discards sampling variations, introducing more space to learn useful features. We theoretically justify this claim under a simple linear autoencoder. Moreover, the implicit decoder offers a rich space to design suitable implicit representations for different tasks. We demonstrate the usefulness of IAE across various self-supervised learning tasks for both 3D objects and 3D scenes. Experimental results show that IAE consistently outperforms the state-of-the-art in each task. Our code will be available at https://github.com/SimingYan/IAE.

show abstract

Section: Methods Modelnet40mentioning

confidence: 99%

Implicit Autoencoder for Point Cloud Self-supervised Representation Learning

Shen¹,

Yang²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…PCN uses an encoder similar to PointNet to extract the global feature from the input point cloud directly, and then employs a decoder to infer the complete point cloud from the global feature. More recent works [1,8,11,13,14,17,25,35,38,40,44,50] have made efforts to preserve the observed geometric details from the local features in incomplete inputs. NSFA [49] separately reconstruct the unknown and known parts.…”

Section: Related Workmentioning

confidence: 99%

“…Then we develop the variations of (a) ∂L com /∂θ all and (b) ∂L match /∂θ all to replace ∂L com /∂θ com or ∂L match /∂θ com in Eq. (11).…”

Section: Analysis Of Different Training Strategiesmentioning

confidence: 99%

KTNet: Knowledge Transfer for Unpaired 3D Shape Completion

Cao¹,

Zhang²,

Xu³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Unpaired 3D object completion aims to predict a complete 3D shape from an incomplete input without knowing the correspondence between the complete and incomplete shapes during training. To build the correspondence between two data modalities, previous methods usually apply adversarial training to match the global shape features extracted by the encoder. However, this ignores the correspondence between multi-scaled geometric information embedded in the pyramidal hierarchy of the decoder, which makes previous methods struggle to generate high-quality complete shapes. To address this problem, we propose a novel unpaired shape completion network, named MFM-Net, using multi-stage feature matching, which decomposes the learning of geometric correspondence into multi-stages throughout the hierarchical generation process in the point cloud decoder. Specifically, MFM-Net adopts a dual path architecture to establish multiple feature matching channels in different layers of the decoder, which is then combined with the adversarial learning to merge the distribution of features from complete and incomplete modalities. In addition, a refinement is applied to enhance the details. As a result, MFM-Net makes use of a more comprehensive understanding to establish the geometric correspondence between complete and incomplete shapes in a localto-global perspective, which enables more detailed geometric inference for generating high-quality complete shapes. We conduct comprehensive experiments on several datasets, and the results show that our method outperforms previous methods of unpaired point cloud completion with a large margin.

show abstract

“…In the field of 3D computer vision, there are many studies concerning various representation form of 3D shapes (e.g. voxels [2,17,29], view [3][4][5][6]8] and point cloud [7,15,32]), and in this paper we concern the segmentation task on the specific form of point cloud. Instance segmentation.The studies concerning 3D instance segmentation can be roughly divided into two directions.…”

Section: Related Workmentioning

confidence: 99%

“…In the field of 3D semantic segmentation, great progress has been made in recent year because of the fast development of deep learning framework. Following the convolutional structure of PointNet [20] and PointNet++ [21], many successors [7,12,14,37] investigate the convolution operations which aggregate the neighbors of a given point by edge attributes in the local region graph. In order to extract the rich representation of contextual relationships between object parts, SPG [10] is proposed to adopt the super-point graph for capture the spatial organization of 3D point clouds, in which a partition of the scanned scene is transformed into geometrically homogeneous elements, and can be further exploited by a graph convolutional network.…”

Section: Related Workmentioning

confidence: 99%