P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds

Abstract

Towards 3D object tracking in point clouds, a novel point-to-box network termed P2B is proposed in an endto-end learning manner. Our main idea is to first localize potential target centers in 3D search area embedded with target information. Then point-driven 3D target proposal and verification are executed jointly. In this way, the time-consuming 3D exhaustive search can be avoided. Specifically, we first sample seeds from the point clouds in template and search area respectively. Then, we execute permutation-invariant feature augmentation to embed target clues from template into search area seeds and represent them with target-specific features. Consequently, the augmented search area seeds regress the potential target centers via Hough voting. The centers are further strengthened with seed-wise targetness scores. Finally, each center clusters its neighbors to leverage the ensemble power for joint 3D target proposal and verification. We apply PointNet++ as our backbone and experiments on KITTI tracking dataset demonstrate P2B's superiority (∼10%'s improvement over state-of-the-art). Note that P2B can run with 40FPS on a single NVIDIA 1080Ti GPU. Our code and model are available at https://github.com/HaozheQi/P2B.

Learning the Incremental Warp for 3D Vehicle Tracking in LiDAR Point Clouds

“…Apart from SiamTrack3D, we compared with other methods-AVODTrack [14], P2B [16], SiamTrack3D-RPN [52], and ICP&TDS-on the testing set. AVODTrack is a tracking-bydetection method which evolved from an advanced 3D detector, AVOD [14], by equipping it with an online association algorithm.…”

“…Specially for 3D tracking, Giancola et al [2] introduced completion regularization to train a Siamese network. Subsequently, in light of the limitation of candidate box generation, Qi et al [16] designed a point-to-box network, Zou [40] reduced redundant search space using a 3D frustum, and Fang et al extended the region proposal network into pointNet++ [41] for 3D tracking. Nevertheless, all of above methods put more emphasis on distinguishing the target from a lot of proposals.…”

“…Consequently, starting from source point clouds, Giancola et al [2] proposed SiamTrack3D to learn a generically template matching function which is trained by the shape completion regularization. Qi et al [16] leveraged deep Hough voting [17] to produce potential bounding boxes. It is worth mentioning that the aforementioned approaches merely focus on determining the best proposal from a set of object proposals.…”

3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud

“…Besides, they did not focus on more effective cross-correlation methods to fuse template and search features. Similarly, Qi and Feng [12] used a Siamese Network to solve 3D object tracking based on VoteNet [13]. They first fused the template and search seeds with a specific approach, then they used VoteNet to generate potential object centers(votes) and estimated position and orientation of the target center based on those votes.…”

“…In practice, these problems limit the performance of the tracking methods. In recent years, there are also some methods [9]- [12] trying to focus on the tracking ability to avoid the problems of the tracking-by-detection framework. For example, a 3D object tracking network is proposed in [9] by mainly utilizing a 3D Siamese tracking network with a Shape Completion network.…”