Coming Down to Earth: Satellite-to-Street View Synthesis for Geo-Localization

Abstract

Figure 1: For a collection of satellite and street images, our method synthesizes the street view for each satellite input (right). It also simultaneously determines the geographic location of a query street image by matching it with the closest satellite image in the database (left→right). This is done in one single architecture which allows for end-to-end training.

LENS: Localization enhanced by NeRF synthesis

“…Meshs built from crowd sourced images usually fail to describe low textured or crowded part of the scene, such as sky and ground. Synthetic images can also be rendered thanks to Generative Adversarial Networks [22,24]. Finally, recent methods that learn a continuous volumetric representation of the scene such as Neural Radiance Fields [1] outperform prior work and exhibit photorealistic results.…”

A graph-matching approach for cross-view registration of over-view and street-view based point clouds

“…Although there have been attempts to address similar problems such as cross-view localization (Castaldo et al, 2015;Hu et al, 2018;Liu and Li, 2019) (i.e. localization on an over-view image given a ground-view) and cross-view synthesis (Regmi and Borji, 2018;Zhai et al, 2017;Lu et al, 2020;Toker et al, 2021) (simulating over-view using ground-view or vice versa), robust cross-view data co-registration appears to be more challenging and has not yet been well investigated. In general, coregistration of such over-view and street-view data encounters three major challenges:…”