Range–Visual–Inertial Odometry with Coarse-to-Fine Image Registration Fusion for UAV Localization

Abstract

In Global Navigation Satellite System (GNSS)-denied environments, image registration has emerged as a prominent approach to utilize visual information for estimating the position of Unmanned Aerial Vehicles (UAVs). However, traditional image-registration-based localization methods encounter limitations, such as strong dependence on the prior initial position information. In this paper, we propose a systematic method for UAV geo-localization. In particular, an efficient range–visual–inertial odometry (RVIO) is proposed to provide local tracking, which utilizes measurements from a 1D Laser Range Finder (LRF) to suppress scale drift in the odometry. To overcome the differences in seasons, lighting conditions, and other factors between satellite and UAV images, we propose an image-registration-based geo-localization method in a coarse-to-fine manner that utilizes the powerful representation ability of Convolutional Neural Networks (CNNs). Furthermore, to ensure the accuracy of global optimization, we propose an adaptive weight assignment method based on the evaluation of the quality of image-registration-based localization. The proposed method is extensively evaluated in both synthetic and real-world environments. The results demonstrate that the proposed method achieves global drift-free estimation, enabling UAVs to accurately localize themselves in GNSS-denied environments.

Geolocalization from Aerial Sensing Images Using Road Network Alignment

“…The geolocalization accuracy was measured using the total translation error in the X (longitude) and Y (latitude) directions between the estimated geographic position and the ground truth. The performance of the proposed method was compared with the commonly used position estimation method [2][3][4]12,21], where the projection point of the center point in the aerial image under the estimated homography transformtion is used as the position of the camera.…”

“…Ref. [12] estimated the geographic position of UAVs by aligning onboard aerial images to satellite imagery using SuperPoint [13], which is a kind of local descriptor.…”

“…Some works on geolocalization use the translation of the estimated similarity transformation to recover the geographic position of the camera [2][3][4]21], which is equal to computing the projection point of the image center using the estimated transformation. Such methods only work properly when the optical axis of the camera is perpendicular to the ground, or external information is used to compensate for the deviation [12]. Some other works [18][19][20] donated the geolocalization result using the homography or simplified similarity transformation between the aerial image and the georeferenced map, wherein the geographic position of the UAV was thus unavailable.…”

Real-Time Optimal States Estimation with Inertial and Delayed Visual Measurements for Unmanned Aerial Vehicles

“…However, the downside is that GPS is susceptible to weather and terrain conditions, and it expends significant power. Laser range and visual-based sensors were used in [4] to obtain accurate position information, and in [5], a laser range sensor combined with visual-inertial odometry was proposed to help complete accurate positioning. However, laser range sensors also have some disadvantages, including their limited perception range and their excessive weight for UAVs.…”

A Binary Fast Image Registration Method Based on Fusion Information

“…Due to the poor stability of UAVs and their vulnerability to interference from the external environment and other factors, airborne aerial images often have problems such as changes in brightness, geometric distortion, atmospheric turbulence, and motion blur; therefore, it is necessary to design a stable and effective image registration method that has good robustness and invariance against lighting, distortion, blur, etc. [5][6][7]. At the same time, aerial image stitching requires a lot of time, so the speed and efficiency of the registration method should also be considered [7].…”