Structure aware 3D single object tracking of point cloud

Zhou, Xiaoyu; Wang, Ling; Yuan, Zhian; Xu, Ke; Ma, Yanxin

doi:10.1117/1.jei.30.4.043010

Cited by 8 publications

(22 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…P2B [21] matches search and template features with cosine similarity and employs Hough Voting [19] to predict the current location. SA-P2B [39] proposes to learn the object structure as an auxiliary task. 3D-SiamRPN [7] uses a RPN [22] head to predict the final results.…”

Section: Related Workmentioning

confidence: 99%

“…Feature Extraction. Following previous methods [7,9,21,39], we employ PointNet++ [20] as the backbone to extract multi-scale point features from the template and the search. However, important information loss may occur during random subsampling in the original PointNet++.…”

Section: System Overviewmentioning

confidence: 99%

“…Attention-based Feature Matching. Different from previous methods that often use cosine similarity [9,21,39] or linear correlation [7] for matching the template and the search, we utilize novel attention operations and propose Point Relation Transformer (PRT). PRT first utilizes a selfattention operation to adaptively aggregate point features for the template and the search area individually, and then performs feature matching with a cross-attention operation.…”

Section: System Overviewmentioning

confidence: 99%

“…Without us-ing many candidates, P2B [21] first use cosine similarity to fuse features of the search region with the template and then adopts the prediction head of VoteNet [19] to generate the final prediction. Following P2B, SA-P2B [39] adds an extra auxiliary network to predict the object structure. In a similar framework, 3D-SiamRPN [7] uses a cross-correlation module for feature matching and an RPN head for final prediction.…”

Section: Introductionmentioning

confidence: 99%

“…In a similar framework, 3D-SiamRPN [7] uses a cross-correlation module for feature matching and an RPN head for final prediction. These methods [9,21,39] essentially perform a linear matching process between features in the search domain and the template, which cannot adapt to different 3D observations caused by random noise, sparsity, and occlusions. Moreover, the inclusion of complex prediction heads as in detection models highly limits their tracking speed, which is a crucial factor for online applications.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

PTTR: Relational 3D Point Cloud Object Tracking with Transformer

Zhou¹,

Luo²,

Luo³

et al. 2021

Preprint

View full text Add to dashboard Cite

In a point cloud sequence, 3D object tracking aims to predict the location and orientation of an object in the current search point cloud given a template point cloud. Motivated by the success of transformers, we propose Point Tracking TRansformer (PTTR), which efficiently predicts high-quality 3D tracking results in a coarse-to-fine manner with the help of transformer operations. PTTR consists of three novel designs. 1) Instead of random sampling, we design Relation-Aware Sampling to preserve relevant points to given templates during subsampling. 2) Furthermore, we propose a Point Relation Transformer (PRT) consisting of a self-attention and a cross-attention module. The global self-attention operation captures long-range dependencies to enhance encoded point features for the search area and the template, respectively. Subsequently, we generate the coarse tracking results by matching the two sets of point features via cross-attention. 3) Based on the coarse tracking results, we employ a novel Prediction Refinement Module to obtain the final refined prediction. In addition, we create a large-scale point cloud single object tracking benchmark based on the Waymo Open Dataset. Extensive experiments show that PTTR achieves superior point cloud tracking in both accuracy and efficiency. Code will be made available at https://github.com/Jasonkks/PTTR.

show abstract

Section: Related Workmentioning

confidence: 99%