Are These from the Same Place? Seeing the Unseen in Cross-View Image Geo-Localization

Rodrigues, Royston; Tani, Masanori

doi:10.1109/wacv48630.2021.00380

Cited by 34 publications

(13 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cross-view geo-localization refers to the problem of matching a given ground image against a dataset of georegistered aerial images to find the geo-location of the vehicle. Current state-of-the-art approaches [4], [5], [6] achieve high recall on corresponding benchmark datasets like CVUSA [7] and CVACT [8]. Such models can also be used to perform coarse geo-tracking (e.g.…”

Section: Cross-view Geo-localizationmentioning

confidence: 99%

“…The similarity function S is defined as the average similarity of the features stored per valid pixel and measures the confidence assigned to the hypothesis h as shown in (5).…”

Section: Feature Map Alignmentmentioning

confidence: 99%

“…(2) Geo-tracking methods start from a known pose and continuously track the vehicle's movement on aerial images or semantic maps provided by geographic information systems (GIS). While state-of-the-art geo-localization approaches [4], [5], [6] have high recall on benchmark datasets [7], [8] they suffer from low metric accuracy (e.g. when employed in a Markov localization scheme [9]).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Continuous Self-Localization on Aerial Images Using Visual and Lidar Sensors

Fervers¹,

Bullinger²,

Bodensteiner³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper proposes a novel method for geo-tracking, i.e. continuous metric self-localization in outdoor environments by registering a vehicle's sensor information with aerial imagery of an unseen target region. Geo-tracking methods offer the potential to supplant noisy signals from global navigation satellite systems (GNSS) and expensive and hard to maintain prior maps that are typically used for this purpose. The proposed geo-tracking method aligns data from on-board cameras and lidar sensors with geo-registered orthophotos to continuously localize a vehicle. We train a model in a metric learning setting to extract visual features from ground and aerial images. The ground features are projected into a top-down perspective via the lidar points and are matched with the aerial features to determine the relative pose between vehicle and orthophoto.Our method is the first to utilize on-board cameras in an end-to-end differentiable model for metric self-localization on unseen orthophotos. It exhibits strong generalization, is robust to changes in the environment and requires only geo-poses as ground truth. We evaluate our approach on the KITTI-360 dataset and achieve a mean absolute position error (APE) of 0.94m. We further compare with previous approaches on the KITTI odometry dataset and achieve state-of-the-art results on the geo-tracking task.

show abstract

Section: Cross-view Geo-localizationmentioning

confidence: 99%

“…The similarity function S is defined as the average similarity of the features stored per valid pixel and measures the confidence assigned to the hypothesis h as shown in (5).…”

Section: Feature Map Alignmentmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Continuous Self-Localization on Aerial Images Using Visual and Lidar Sensors

Fervers¹,

Bullinger²,

Bodensteiner³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…PL-Net [21] introduces a part loss to realize automatic detection of various parts of the human body, thereby increasing the discrimination on unseen persons. Rodrigues et al [42] addressed the temporal gap between scenes by proposing a semantically driven data augmentation technique that gives Siamese networks the ability to hallucinate unseen objects, and then apply a multi-scale attentive embedding network to perform matching tasks. Our proposed FSRA is also one of the part-based methods which is inspired by the LPN, the difference is that we do not add additional supervision but achieve automatic region segmentation, which makes our FSRA have excellent robustness and resistance to position shift.…”

Section: A Cross-view Geo-localizationmentioning

confidence: 99%

A Transformer-Based Feature Segmentation and Region Alignment Method For UAV-View Geo-Localization

Dai,

Hu,

Zhuang

et al. 2022

Preprint

View full text Add to dashboard Cite

Cross-view geo-localization is a task of matching the same geographic image from different views, e.g., unmanned aerial vehicle (UAV) and satellite. The most difficult challenges are the position shift and the uncertainty of distance and scale. Existing methods are mainly aimed at digging for more comprehensive fine-grained information. However, it underestimates the importance of extracting robust feature representation and the impact of feature alignment. The CNN-based methods have achieved great success in cross-view geo-localization. However it still has some limitations, e.g., it can only extract part of the information in the neighborhood and some scale reduction operations will make some fine-grained information lost. In particular, we introduce a simple and efficient transformer-based structure called Feature Segmentation and Region Alignment (FSRA) to enhance the model's ability to understand contextual information as well as to understand the distribution of instances. Without using additional supervisory information, FSRA divides regions based on the heat distribution of the transformer's feature map, and then aligns multiple specific regions in different views one on one. Finally, FSRA integrates each region into a set of feature representations. The difference is that FSRA does not divide regions manually, but automatically based on the heat distribution of the feature map. So that specific instances can still be divided and aligned when there are significant shifts and scale changes in the image. In addition, a multiple sampling strategy is proposed to overcome the disparity in the number of satellite images and that of images from other sources. Experiments show that the proposed method has superior performance and achieves the state-of-the-art in both tasks of drone view target localization and drone navigation. Code will be released at https://github.com/Dmmm1997/FSRA

show abstract

“…Current works on cross-view geolocalization follow image based approach since the existing datasets only contain image pairs for ground and aerial view [25,16,9,31,18]. However, some papers do report testing their model on videos using a frame-by-frame approach [9].…”

Section: Related Workmentioning

confidence: 99%

GAMa: Cross-view Video Geo-localization

Vyas¹,

Chen²,

Shah³

2022

Preprint

View full text Add to dashboard Cite

The existing work in cross-view geo-localization is based on images where a ground panorama is matched to an aerial image. In this work, we focus on ground videos instead of images which provides additional contextual cues which are important for this task. There are no existing datasets for this problem, therefore we propose GAMa dataset, a large-scale dataset with ground videos and corresponding aerial images. We also propose a novel approach to solve this problem. At clip-level, a short video clip is matched with corresponding aerial image and is later used to get video-level geo-localization of a long video. Moreover, we propose a hierarchical approach to further improve the clip-level geolocalization. It is a challenging dataset, unaligned and limited field of view, and our proposed method achieves a Top-1 recall rate of 19.4% and 45.1% @1.0mile. Code and dataset are available at following link.

show abstract

Are These from the Same Place? Seeing the Unseen in Cross-View Image Geo-Localization

Cited by 34 publications

References 23 publications

Continuous Self-Localization on Aerial Images Using Visual and Lidar Sensors

Continuous Self-Localization on Aerial Images Using Visual and Lidar Sensors

A Transformer-Based Feature Segmentation and Region Alignment Method For UAV-View Geo-Localization

GAMa: Cross-view Video Geo-localization

Contact Info

Product

Resources

About