The 3D Position Estimation and Tracking of a Surface Vehicle Using a Mono-Camera and Machine Learning

Abstract

The ability to obtain the 3D position of target vehicles is essential to managing and coordinating a multi-robot operation. We investigate an ML-backed object localization and tracking system to estimate the target’s 3D position based on a mono-camera input. The passive vision-only technique provides a robust field awareness in challenging conditions such as GPS-denied or radio-silent environments. Our processing pipeline utilizes a YOLOv5 neural network as the back-end detection module and a temporal filtering technique to improve detection and tracking accuracy. The filtering process effectively removes false positive labels to improve tracking accuracy. We propose a piecewise projection model to predict the target 3D position from the estimated 2D bounding box. Our projection model utilizes the co-plane property of ground vehicles to calculate 2D–3D mapping. Experimental results show that the piecewise model is more accurate than existing methods when the training dataset is not evenly distributed in the sampling space. Our piecewise model outperforms the singular RANSAC-based and the 6DPose methods by 28% in location errors. A less than 10-m error is observed for most near-to-mid-range cases.

A Low-Cost Relative Positioning Method for UAV/UGV Coordinated Heterogeneous System Based on Visual-Lidar Fusion

“…Chang et al [20] focused on the development of a proactive guidance system for accurate UAV landing on a dynamic platform using a visual-inertial approach. Additionally, a mono-camera and machine learning were used to estimate and track the 3D position of a surface vehicle [21]. These studies highlight the potential of learning-based methods in visual positioning tasks and demonstrate their applicability in various domains.…”

“…Then, the Kalman gain (K k ) is computed by using Equation (20). The optimal state (ŝ k ) at time k can be acquired in Equation (21) with K k and ŝ− k . Finally, Equation ( 22) corrects the corresponding covariance matrix (P k ) of the state at time k for the calculation of next time step (k + 1).…”