Cooperative Multi-Sensor Tracking of Vulnerable Road Users in the Presence of Missing Detections

Abstract

This paper presents a vulnerable road user (VRU) tracking algorithm capable of handling noisy and missing detections from heterogeneous sensors. We propose a cooperative fusion algorithm for matching and reinforcing of radar and camera detections using their proximity and positional uncertainty. The belief in the existence and position of objects is then maximized by temporal integration of fused detections by a multi-object tracker. By switching between observation models, the tracker adapts to the detection noise characteristics making it robust to individual sensor failures. The main novelty of this paper is an improved imputation sampling function for updating the state when detections are missing. The proposed function uses a likelihood without association that is conditioned on the sensor information instead of the sensor model. The benefits of the proposed solution are two-fold: firstly, particle updates become computationally tractable and secondly, the problem of imputing samples from a state which is predicted without an associated detection is bypassed. Experimental evaluation shows a significant improvement in both detection and tracking performance over multiple control algorithms. In low light situations, the cooperative fusion outperforms intermediate fusion by as much as 30%, while increases in tracking performance are most significant in complex traffic scenes.

Signal Processing Platform for Long-Range Multi-Spectral Electro-Optical Systems

“…The described VVSP platform has more general application than that of long-range surveillance systems. Virtually any multi-sensor system can utilize this platform, especially in sensor fusion applications such as the integration of radars and multiple cameras for target tracking [ 41 ]. Similarly, autonomous vehicle applications like those presented in [ 42 ] can utilize the VVSP platform without significant modifications.…”

Perception System based on Cooperative Fusion of Lidar and Cameras

“…delta-Dirac mass located in r (i) . For more details on the tracker implementation we refer the reader to our previous work in [8], [9].…”