Vision-Based Target Geolocation and Optimal Surveillance on an Unmanned Aerial Vehicle

Abstract

A real-time computer vision algorithm for the identification and geolocation of ground targets was developed and implemented on the Penn State University / Applied Research Laboratory Unmanned Aerial Vehicle (PSU/ARL UAV) system. The geolocation data is filtered using a linear Kalman filter, which provides a smoothed estimate of target location and target velocity. The vision processing routine and estimator are coupled with an onboard path planning algorithm that optimizes the vehicle trajectory to maximize surveillance coverage of the targets.The vision processing and estimation routines were flight tested onboard a UAV system with a human pilot-in-the-loop. It was found that GPS latency had a significant effect on the geolocation error, and performance was significantly improved when using latency compensation. The combined target geolocation and path planning system was tested on the ground using a hardware-in-the-loop simulation, and resulted successful tracking and observation of a fixed target. Timing results showed that is is feasible to implement total system in real time.

Landmark Fingerprinting and Matching for Aerial Positioning Systems

“…Because the system only has approximate information about the current position estimate (and is designed be able to operate without any such information), it is not appropriate to use an approach where individual landmarks in the image are projected onto the ground and the estimated coordinates are used for matching [13]. This presents a challenge because the descriptor can only use information contained within the image to describe each landmark.…”

Fusion of Unmanned Aerial Vehicle Range and Vision Sensors Using Fuzzy Logic and Particles

“…The use of color filtering to identify large red balls is discussed in Ross et al 2 In the reference an image is converted to the Hue-Saturation-Value (HSV) color space. The image is then filtered in order to find pixels that have a hue below 30 degrees or above 315 degrees (hue wraps around the color scale) and a saturation above 0.3 (this excludes pixels that are "washed out").…”

Efficient target geolocation by highly uncertain small air vehicles

“…Various formulations of the Kalman filter have also been applied [8], [9], [10] with varying success, but such attempts fall short due to either using arbitrary observation uncertainties or by relying on linearization of the ray intersection function (which will be shown to be a poor approximation given realistic UAV state error), resulting in filter inconsistency.…”

“…This leads in practice to requiring a very large number of particles for accurate geolocation and to avoid particle depletion, resulting in poor performance on embedded hardware or when used for predictive planning requiring frequent likelihood evaluation. Given the dominant heading uncertainty observed in the literature [6], [8], [13], a polar representation may be more appropriate, and inspiration may be drawn from efforts at range-only localization [14].…”