3DViewGraph: Learning Global Features for 3D Shapes from A Graph of Unordered Views with Attention

Abstract

Learning global features by aggregating information over multiple views has been shown to be effective for 3D shape analysis. For view aggregation in deep learning models, pooling has been applied extensively. However, pooling leads to a loss of the content within views, and the spatial relationship among views, which limits the discriminability of learned features. We propose 3DViewGraph to resolve this issue, which learns 3D global features by more effectively aggregating unordered views with attention. Specifically, unordered views taken around a shape are regarded as view nodes on a view graph. 3DViewGraph first learns a novel latent semantic mapping to project low-level view features into meaningful latent semantic embeddings in a lower dimensional space, which is spanned by latent semantic patterns. Then, the content and spatial information of each pair of view nodes are encoded by a novel spatial pattern correlation, where the correlation is computed among latent semantic patterns. Finally, all spatial pattern correlations are integrated with attention weights learned by a novel attention mechanism. This further increases the discriminability of learned features by highlighting the unordered view nodes with distinctive characteristics and depressing the ones with appearance ambiguity. We show that 3DViewGraph outperforms state-of-theart methods under three large-scale benchmarks.

Generative Enhancement of 3D Image Classifiers

“…State of the art generative [13][14][15] and non-generative [16][17][18] classifiers compete very closely in terms of 3D image classification accuracy, as can be seen in the results of benchmarks, such as ModelNet [19]. While models that utilize the two approaches differ significantly in structure, we see the potential for their combination by enhancing the non-generative classifiers with elements of generative modeling.…”

“…Generally, these classification methods can be grouped into view-based and volume, or shape-based techniques, depending on the form of data that are present at the input of the classifier. Many of the most successful classifiers [16][17][18]38] are based on the multi-view approach, which renders or acquires several views of the classified object and extract features from those 2D projections. Efficient aggregation of the information from multiple views is the key to high classification accuracy.…”

“…Efficient aggregation of the information from multiple views is the key to high classification accuracy. This problem is explored in [17], where the authors process the views as an unordered graph by learning a semantic feature mapping and aggregating the patterns using a novel pair-wise spatial pattern correlation and a multi-attention model. Similarly, in [18], the authors learn global 3D features from multiple views by segmenting the projections into semantically meaningful parts.…”

Multi-Angle Point Cloud-VAE: Unsupervised Feature Learning for 3D Point Clouds From Multiple Angles by Joint Self-Reconstruction and Half-to-Half Prediction

“…Deep learning models have led to significant progress in feature learning for 3D shapes [13,12,15,14,18,19,10,20,16,11]. Here, we focus on reviewing studies on point clouds.…”

CF-SIS: Semantic-Instance Segmentation of 3D Point Clouds by Context Fusion with Self-Attention

“…In the field of 3D computer vision, there are many studies concerning various representation form of 3D shapes (e.g. voxels [2,17,29], view [3][4][5][6]8] and point cloud [7,15,32]), and in this paper we concern the segmentation task on the specific form of point cloud. Instance segmentation.The studies concerning 3D instance segmentation can be roughly divided into two directions.…”