Global Assists Local: Effective Aerial Representations for Field of View Constrained Image Geo-Localization

Rodrigues, Royston; Tani, Masanori

doi:10.1109/wacv51458.2022.00275

Cited by 19 publications

(5 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zemene et al (2019) designed a retrieval method for querying in a city-wide reference image database with known absolute coordinates, thereby determining the geo-location of the query image. Similarly, Rodrigues and Tani (2022) performed retrieval between ground images and a large geotagged aerial image database. Some methods enhance the retrieval capabilities of the network by improving training strategies or modifying the framework of the model.…”

Section: Cross-view Remote Sensing Image Retrievalmentioning

confidence: 99%

A Transformer and Visual Foundation Model-Based Method for Cross-View Remote Sensing Image Retrieval

Yin,

Ye,

Luo

2024

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. Retrieving UAV images that lack POS information with georeferenced satellite orthoimagery is challenging due to the differences in angles of views. Most existing methods rely on deep neural networks with a large number of parameters, leading to substantial time and financial investments in network training. Consequently, these methods may not be well-suited for downstream tasks that have high timeliness requirements. In this work, we propose a cross-view remote sensing image retrieval method based on transformer and visual foundation model. We investigated the potential of visual foundation model for extracting common features from cross-view images. Training is only conducted on a small, self-designed retrieval head, alleviating the burden of network training. Specifically, we designed a CVV module to optimize the features extracted from the visual foundation model, making these features more adept for cross-view image retrieval tasks. And we designed an MLP head to achieve similarity discrimination. The method is verified on a publicly available dataset containing multiple scenes. Our method shows excellent results in terms of both efficiency and accuracy on 15 sub-datasets (10 or 50 scene categories) derived from the public dataset, which holds practical value in engineering applications with streamlined scene categories and constrained computational resources. Furthermore, we initiated a comprehensive discussion and conducted ablation experiments on the network design to validate its efficacy. Additionally, we analyzed the presence of overfitting within the network and deliberated on the limitations of our study, proposing potential avenues for future enhancements.

show abstract

Section: Cross-view Remote Sensing Image Retrievalmentioning

confidence: 99%

A Transformer and Visual Foundation Model-Based Method for Cross-View Remote Sensing Image Retrieval

Yin,

Ye,

Luo

2024

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…DSM-Net [41] utilizes dynamic similarity to align image directions within a limited field of view. In another study [42], the robustness was further enhanced by considering the local and global properties of aerial images based on DSM-Net. Similarly, the coarse positioning performance was significantly improved by considering the geometric correspondence of feature points based on DSM-Net [43].…”

Section: A Cnns In Geo-localizationmentioning

confidence: 99%

GeoFormer: An Effective Transformer-Based Siamese Network for UAV Geolocalization

Li,

Yang,

Fan

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Cross-view geo-localization of unmanned aerial vehicles (UAVs) is a challenging task due to the positional discrepancies and uncertainties in scale and distance between UAVs and satellite views. Existing transformer-based geolocalization methods mainly use encoders to mine image contextual information. However, these methods have some limitations when dealing with scale changes in cross-view images. Therefore, we present an effective transformer-based Siamese network tailored for UAV geo-localization, called GeoFormer. Firstly, an efficient transformer feature extraction network was designed, which utilizes linear attention to reduce the computational complexity and improve the computational efficiency of the network. Among them, we designed an efficient separable perceptron module based on depth-wise separable convolution, which can effectively reduce the computational cost while improving the feature representation of the network. Secondly, we proposed a multi-scale feature aggregation module (MFAM), which deeply fuses salient features at different scales through a feed-forward neural network to generate global feature representations with rich semantics, which improves the model's ability to capture image details and represent robust features. Additionally, we designed a semantic-guided region segmentation module (SRSM), which utilizes a k-modes clustering algorithm to divide the feature map into multiple regions with semantic consistency and performs feature recognition within each semantic region to improve the accuracy of image matching. Finally, we designed a hierarchical reinforcement rotation matching strategy to achieve accurate UAV geo-localization based on the retrieval results of UAV view query satellite images using SuperPoint keypoints extraction and LightGlue rotation matching. According to the experimental results, our method effectively achieves UAV geo-localization.

show abstract

“…Nonetheless, its application in cross-view geo-localization is not fully explored due to the fragility of the spatial correspondence between aerial and ground images which can be easily disrupted by even minor perturbations. Some existing methods attempt to address this issue by randomly rotating or shifting one view while fixing the other one [3], [15], [21], [29]. Alternatively, [30] randomly blackout ground objects according to their segmentation from street images.…”

Section: Related Workmentioning

confidence: 99%

“…Nonetheless, data augmentation in cross-view geo-localization is very limited due to the vulnerability of the spatial correspondence between aerial and ground images which can be easily sabotaged by a minor interference. For instance, most existing methods (Liu and Li 2019;Rodrigues and Tani 2022;Vo and Hays 2016;Cai et al 2019) randomly rotate or shift one view while fixing the other one. On the other hand, Rodrigues and Tani (2021) randomly blackout ground objects according to their segmentation from street images.…”

Section: Cross-view Geo-localizationmentioning

confidence: 99%

Cross-View Geo-Localization via Learning Disentangled Geometric Layout Correspondence

Zhang

Sultani

et al. 2023

AAAI

View full text Add to dashboard Cite

Cross-view geo-localization aims to estimate the location of a query ground image by matching it to a reference geo-tagged aerial images database. As an extremely challenging task, its difficulties root in the drastic view changes and different capturing time between two views. Despite these difficulties, recent works achieve outstanding progress on cross-view geo-localization benchmarks. However, existing methods still suffer from poor performance on the cross-area benchmarks, in which the training and testing data are captured from two different regions. We attribute this deficiency to the lack of ability to extract the spatial configuration of visual feature layouts and models' overfitting on low-level details from the training set. In this paper, we propose GeoDTR which explicitly disentangles geometric information from raw features and learns the spatial correlations among visual features from aerial and ground pairs with a novel geometric layout extractor module. This module generates a set of geometric layout descriptors, modulating the raw features and producing high-quality latent representations. In addition, we elaborate on two categories of data augmentations, (i) Layout simulation, which varies the spatial configuration while keeping the low-level details intact. (ii) Semantic augmentation, which alters the low-level details and encourages the model to capture spatial configurations. These augmentations help to improve the performance of the cross-view geo-localization models, especially on the cross-area benchmarks. Moreover, we propose a counterfactual-based learning process to benefit the geometric layout extractor in exploring spatial information. Extensive experiments show that GeoDTR not only achieves state-of-the-art results but also significantly boosts the performance on same-area and cross-area benchmarks. Our code can be found at https://gitlab.com/vail-uvm/geodtr.

show abstract

Global Assists Local: Effective Aerial Representations for Field of View Constrained Image Geo-Localization

Cited by 19 publications

References 27 publications

A Transformer and Visual Foundation Model-Based Method for Cross-View Remote Sensing Image Retrieval

A Transformer and Visual Foundation Model-Based Method for Cross-View Remote Sensing Image Retrieval

GeoFormer: An Effective Transformer-Based Siamese Network for UAV Geolocalization

Cross-View Geo-Localization via Learning Disentangled Geometric Layout Correspondence

Contact Info

Product

Resources

About