No Blind Spots: Full-Surround Multi-Object Tracking for Autonomous Vehicles Using Cameras and LiDARs

Abstract

Online multi-object tracking (MOT) is extremely important for high-level spatial reasoning and path planning for autonomous and highly-automated vehicles. In this paper, we present a modular framework for tracking multiple objects (vehicles), capable of accepting object proposals from different sensor modalities (vision and range) and a variable number of sensors, to produce continuous object tracks. This work is a generalization of the MDP framework for MOT proposed in [1], with some key extensions -First, we track objects across multiple cameras and across different sensor modalities. This is done by fusing object proposals across sensors accurately and efficiently. Second, the objects of interest (targets) are tracked directly in the real world. This is a departure from traditional techniques where objects are simply tracked in the image plane. Doing so allows the tracks to be readily used by an autonomous agent for navigation and related tasks.To verify the effectiveness of our approach, we test it on real world highway data collected from a heavily sensorized testbed capable of capturing full-surround information. We demonstrate that our framework is well-suited to track objects through entire maneuvers around the ego-vehicle, some of which take more than a few minutes to complete. We also leverage the modularity of our approach by comparing the effects of including/excluding different sensors, changing the total number of sensors, and the quality of object proposals on the final tracking result.

Set-Based Prediction of Traffic Participants Considering Occlusions and Traffic Rules

“…We solely focus on motion prediction of other traffic participants [24]- [26], which is an integral part of motion planning [27]- [29] and risk assessment [24], [30]. The following related aspects are beyond the scope of this paper: extracting the information of surrounding traffic participants from sensor measurements [31]- [33], the uncertainty of these measurements [34]- [36], and implications on the prediction for connected vehicles [37], [38].…”

Pedestrian Models for Autonomous Driving Part I: Low-Level Models, From Sensing to Tracking

“…Some works with 3D lidar sensors include [114], [207], [234], which proposed online classification of humans for 3D lidar tracking. In [175], both camera and lidar data are used to improve people tracking. e) MTT with Scene Understanding: Scene understanding can provide contextual information and scene structure for the tracking algorithm, especially in crowded scenes.…”

Ground Plane Polling for 6DoF Pose Estimation of Objects on the Road

“…It is therefore desirable to carry out 3D object detection with monocular cameras, if suitable robustness can be achieved. A robust 3D detector could also in turn improve the performance of purely camera based 3D trackers [1]. This however introduces many challenges, most of which stem from the fact that predicting 3D attributes from 2D measurements is an ill-posed problem.…”