A Practical Cross-View Image Matching Method between UAV and Satellite for UAV-Based Geo-Localization

Abstract

Cross-view image matching has attracted extensive attention due to its huge potential applications, such as localization and navigation. Unmanned aerial vehicle (UAV) technology has been developed rapidly in recent years, and people have more opportunities to obtain and use UAV-view images than ever before. However, the algorithms of cross-view image matching between the UAV view (oblique view) and the satellite view (vertical view) are still in their beginning stage, and the matching accuracy is expected to be further improved when applied in real situations. Within this context, in this study, we proposed a cross-view matching method based on location classification (hereinafter referred to LCM), in which the similarity between UAV and satellite views is considered, and we implemented the method with the newest UAV-based geo-localization dataset (University-1652). LCM is able to solve the imbalance of the input sample number between the satellite images and the UAV images. In the training stage, LCM can simplify the retrieval problem into a classification problem and consider the influence of the feature vector size on the matching accuracy. Compared with one study, LCM shows higher accuracies, and Recall@K (K ∈ {1, 5, 10}) and the average precision (AP) were improved by 5–10%. The expansion of satellite-view images and multiple queries proposed by the LCM are capable of improving the matching accuracy during the experiment. In addition, the influences of different feature sizes on the LCM’s accuracy are determined, and we found that 512 is the optimal feature size. Finally, the LCM model trained based on synthetic UAV-view images was evaluated in real-world situations, and the evaluation result shows that it still has satisfactory matching accuracy. The LCM can realize the bidirectional matching between the UAV-view image and the satellite-view image and can contribute to two applications: (i) UAV-view image localization (i.e., predicting the geographic location of UAV-view images based on satellite-view images with geo-tags) and (ii) UAV navigation (i.e., driving the UAV to the region of interest in the satellite-view image based on the flight record).

SUES-200: A Multi-height Multi-scene Cross-view Image Benchmark Across Drone and Satellite

“…Liu el at. [20] propose LCM utilizes ResNet [27] as the backbone network and trains the image retrieval problem as a classification problem, and uses data augmentation to extend the satellite view images. The results show that LCM's Recall@1 and Fig.…”

“…Zheng et al [19] establish the first drone-based multi-source cross-view matching dataset, University-1652, which contains three views: street view, drone, and satellite, and it also publishes a baseline by designing a multibranch CNN network. [20]- [24]conduct a more in-depth study of University-1652 and significantly improve the accuracy of the matching system. However, University-1652 still has the following problems: 1.University-1652 uses synthetic images of drone views, which lack real-world lighting variations.…”

A Transformer-Based Feature Segmentation and Region Alignment Method For UAV-View Geo-Localization

“…In addition, inspired by LCM [24], we realize that satellite images are highly scarce in the University-1652 [1] datasets, and expanded images can effectively improve network learning capabilities. In view of that, we propose a multiple sampling strategy to expand satellite imagery.…”

“…in LCM [24] (the authors achieved the best using equalmultiplicity sampling of UAV and satellite images).…”

A Faster and More Effective Cross-View Matching Method of UAV and Satellite Images for UAV Geolocalization

“…Multiscale block attention network (MSBA) consists of two major branches, namely, satellite and drone views, and the street-view branch was removed, which is different from other mainstream methods [21,28,48]. Focusing on both performance and efficiency, Resnet50 [49] is used as the backbone to extract image features.…”