FANTrack: 3D Multi-Object Tracking with Feature Association Network

Baser, Erkan; Balasubramanian, Venkateshwaran; Bhattacharyya, Prarthana; Czarnecki, Krzysztof

doi:10.1109/ivs.2019.8813779

Cited by 98 publications

(53 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rigid 3D object motion prediction. While there is a vast amount of works on 3D object reconstruction [51,19,41], detection [9,18,11] and tracking [8,4], only very few approaches address the problem of predicting future rigid motion [6,63,32,59]. Among these, it is worth to mention Byravan et al [6], that predict the future 3D pose given an image of an object and the action being applied to it.…”

Section: Related Workmentioning

confidence: 99%

Context-Aware Human Motion Prediction

Corona

Pumarola

Alenyà

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

136

View full text Add to dashboard Cite

The problem of predicting human motion given a sequence of past observations is at the core of many applications in robotics and computer vision. Current state-ofthe-art formulate this problem as a sequence-to-sequence task, in which a historical of 3D skeletons feeds a Recurrent Neural Network (RNN) that predicts future movements, typically in the order of 1 to 2 seconds. However, one aspect that has been obviated so far, is the fact that human motion is inherently driven by interactions with objects and/or other humans in the environment. In this paper, we explore this scenario using a novel context-aware motion prediction architecture. We use a semantic-graph model where the nodes parameterize the human and objects in the scene and the edges their mutual interactions. These interactions are iteratively learned through a graph attention layer, fed with the past observations, which now include both object and human body motions. Once this semantic graph is learned, we inject it to a standard RNN to predict future movements of the human/s and object/s. We consider two variants of our architecture, either freezing the contextual interactions in the future of updating them. A thorough evaluation in the "Whole-Body Human Motion Database" [29] shows that in both cases, our context-aware networks clearly outperform baselines in which the context information is not considered.

show abstract

Section: Related Workmentioning

confidence: 99%

Context-Aware Human Motion Prediction

Corona

Pumarola

Alenyà

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

136

View full text Add to dashboard Cite

show abstract

“…Previous works usually use tracking-by-detection framework to track 3D objects. They first utilize an off-the-shelf detector [32], [33] to detect objects and then use probability methods [6], [34]- [39] to match the detection results overtime. These tracking methods can be divided into two categories according to the detectors used, one-stage(singleshot) detector-based tracking methods [36], [37] and two-stage detector-based tracking methods [38], [39].…”

Section: B 3d Object Trackingmentioning

confidence: 99%

“…They first utilize an off-the-shelf detector [32], [33] to detect objects and then use probability methods [6], [34]- [39] to match the detection results overtime. These tracking methods can be divided into two categories according to the detectors used, one-stage(singleshot) detector-based tracking methods [36], [37] and two-stage detector-based tracking methods [38], [39]. Simon et al [36] used a modified one-stage detector [40] to detect target and used Labelled Multi-Bernoulli Filter to track the target with 100FPS.…”

Section: B 3d Object Trackingmentioning

confidence: 99%

3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud

et al. 2021

View full text Add to dashboard Cite

3D single object tracking is a key issue for autonomous following robot, where the robot should robustly track and accurately localize the target for efficient following. In this paper, we propose a 3D tracking method called 3D-SiamRPN Network to track a single target object by using raw 3D point cloud data. The proposed network consists of two subnetworks. The first subnetwork is feature embedding subnetwork which is used for point cloud feature extraction and fusion. In this subnetwork, we first use PointNet++ to extract features of point cloud from template and search branches. Then, to fuse the information of features in the two branches and obtain their similarity, we propose two cross correlation modules, named Pointcloud-wise and Point-wise respectively. The second subnetwork is region proposal network(RPN), which is used to get the final 3D bounding box of the target object based on the fusion feature from cross correlation modules. In this subnetwork, we utilize the regression and classification branches of a region proposal subnetwork to obtain proposals and scores, thus get the final 3D bounding box of the target object. Experimental results on KITTI dataset show that our method has a competitive performance in both Success and Precision compared to the state-of-the-art methods, and could run in real-time at 20.8 FPS. Additionally, experimental results on H3D dataset demonstrate that our method also has good generalization ability and could achieve good tracking performance in a new scene without re-training. Index Terms-3D single object tracking, LIDAR point-cloud, siamese network, region proposal network. I. INTRODUCTIONO BJECT tracking, which is a key issue in computer vision and robotics, could be used in a wide range of applications, such as augmented reality, self-driving cars and mobile robotics. Traditionally, cameras are widely used for object tracking, known as visual object tracking (VOT). For mobile robots or self-driving cars, 3D object tracking is usually more Manuscript

show abstract

“…Osep et al [46] proposed a 2D-3D Kalman filter to jointly use images and the 3D world coordinate system. Baser et al [47] proposed an online multiobject tracking method based on 2 Mathematical Problems in Engineering CNN. Hu et al [48] used long short-term memory network (LSTM) learning module to predict long-term motion more accurately.…”

Section: D Motmentioning

confidence: 99%

A 3D Multiobject Tracking Algorithm of Point Cloud Based on Deep Learning

Wang¹,

Huang²,

Wang³

et al. 2020

Mathematical Problems in Engineering

View full text Add to dashboard Cite

3D multiobject tracking (MOT) is an important part of road condition detection and hazard warning algorithm in roadside systems and autonomous driving systems. There is a tricky problem in 3D MOT that the identity of occluded object switches after it reappears. Given the good performance of the 2D MOT, this paper proposes a 3D MOT algorithm with deep learning based on the multiobject tracking algorithm. Firstly, a 3D object detector was used to obtain oriented 3D bounding boxes from point clouds. Secondly, a 3D Kalman filter was used for state estimation, and reidentification algorithm was used to match feature similarity. Finally, data association was conducted by combining Hungarian algorithm. Experiments show that the proposed method can still match the original trajectory after the occluded object reappears and run at a rate of 59 FPS, which has achieved advanced results in the existing 3D MOT system.

show abstract

FANTrack: 3D Multi-Object Tracking with Feature Association Network

Cited by 98 publications

References 40 publications

Context-Aware Human Motion Prediction

Context-Aware Human Motion Prediction

3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud

A 3D Multiobject Tracking Algorithm of Point Cloud Based on Deep Learning

Contact Info

Product

Resources

About