Point Cloud Learning with Transformer

Abstract

Remarkable performance from Transformer networks in Natural Language Processing promote the development of these models in dealing with computer vision tasks such as image recognition and segmentation. In this paper, we introduce a novel framework, called Multi-level Multi-scale Point Transformer (MLMSPT) that works directly on the irregular point clouds for representation learning. Specifically, a point pyramid transformer is investigated to model features with diverse resolutions or scales we defined, followed by a multi-level transformer module to aggregate contextual information from different levels of each scale and enhance their interactions. While a multi-scale transformer module is designed to capture the dependencies among representations across different scales. Extensive evaluation on public benchmark datasets demonstrate the effectiveness and the competitive performance of our methods on 3D shape classification, part segmentation and semantic segmentation tasks.

3DCTN: 3D Convolution-Transformer Network for Point Cloud Classification

“…Han et. al [15] proposed another point-wise approach to learn global features. Specifically, they designed a multilevel Transformer to extract global features of target point clouds with different resolutions, followed by concatenating these features and feeding them into a multi-scale Transformer to get the final global features.…”

“…Vector Attention. Usually, there are two kinds of selfattention operators: vector attention and scalar attention, where the later has been applied in many previous 3D Transformer works [13], [15], while the former has been proven to be more effective than other operators in the fields of 2D image [31] and 3D point cloud processing [14].…”

“…Compared to other deep learning-based approaches, our method still achieves the highest mAcc thanks to the strong global feature learning ability of Transformers, and outperforms most of state-of-the-art approaches like Graph Convolutionbased method DGCNN [12], point-based convolution method PointCNN [10], and attention-based method Point2Sequence [38] in terms of OA. [2] 83.0% 85.9% Subvolume [39] 86.0% 89.2% PointNet [6] 86.0% 89.2% PointNet++ [7] -91.9% diffConv [40] 90.4% 93.2% PointWeb [41] 89.4% 92.3% CurveNet [42] 90.4% 93.1% PointCNN [10] 88.1% 92.2% Point2Sequence [38] 90.4% 92.6% DGCNN [12] 90.2% 92.2% FatNet [43] 90.6% 93.2% Transformer-based Methods A-SCN [36] 87.6% 90.0% PATs [44] -91.7% GAPNet [45] 89.7% 92.4% LFT-Net [16] 89.7% 93.2% 3DETR [46] 89.9% 91.9% PointTransformer [47] -92.8% MLMST [15] -92.9% PointCloudTransformer [13] -93.2% PointTransformer [14] 90.6% 93.7% CloudTransformers [48] 90.8% 93.1% CAA [37] 91.0% 93.8% Ours (Single-scale) 91.2% 92.7% Ours (Multi-scale) 91.2% 93.3%…”

“…δ(q 1 , k 1 ) δ(q 1 , k 2 ) ... δ(q 1 , k S ) δ(q 2 , k 1 ) δ(q 2 , k 2 ) ... δ(q 2 , k S ) ... ... ... ... δ(q S , k 1 ) δ(q S , k 2 ) ... δ(q S , k S )     (15) where δ can be expressed as different forms:…”

Transformers in 3D Point Clouds: A Survey

“…In the field of 3D point cloud processing, high-level tasks usually include: classification & segmentation [4], [7], [8], [28]- [30], [33], [35], [36], [38], [40], [41], [47], [82], [88], [89], object detection [31], [42], [43], [49]- [51], [66], [90], [91] and tracking [52]- [54], registration [55]- [59], [68], [69] and so on. Here, we start by introducing classification & segmentation tasks, which are very common and fundamental research topics in the field of 3D computer vision.…”