Self-supervised learning for using overhead imagery as maps in outdoor range sensor localization

Abstract

Traditional approaches to outdoor vehicle localization assume a reliable, prior map is available, typically built using the same sensor suite as the on-board sensors used during localization. This work makes a different assumption. It assumes that an overhead image of the workspace is available and utilizes that as a map for use for range-based sensor localization by a vehicle. Here, range-based sensors are radars and lidars. Our motivation is simple, off-the-shelf, publicly available overhead imagery such as Google satellite images can be a ubiquitous, cheap, and powerful tool for vehicle localization when a usable prior sensor map is unavailable, inconvenient, or expensive. The challenge to be addressed is that overhead images are clearly not directly comparable to data from ground range sensors because of their starkly different modalities. We present a learned metric localization method that not only handles the modality difference, but is also cheap to train, learning in a self-supervised fashion without requiring metrically accurate ground truth. By evaluating across multiple real-world datasets, we demonstrate the robustness and versatility of our method for various sensor configurations in cross-modality localization, achieving localization errors on-par with a prior supervised approach while requiring no pixel-wise aligned ground truth for supervision at training. We pay particular attention to the use of millimeter-wave radar, which, owing to its complex interaction with the scene and its immunity to weather and lighting conditions, makes for a compelling and valuable use case.

Continuous Self-Localization on Aerial Images Using Visual and Lidar Sensors

“…Missing cells indicate scenes that are not evaluated or not tracked successfully. Tang et al [14], [15], [16] show results for x and y dimensions separately -we report x 2 + y 2 . For each method, the best reported results are shown.…”

“…Previous methods based on end-to-end trainable models [14], [15], [37], [16] utilize only input from range-scanners (i.e. radar and lidar) on the ground vehicle.…”

“…radar and lidar) on the ground vehicle. The works by Tang et al [14], [15], [16] also discard lidar points on the ground plane and thereby focus entirely on vertical structures.…”

“…• Several methods [13], [14], [15], [16] train and test on data from the same city area. Others [17], [18] assume that the ground region or aerial data to be tested on has already been seen during the training stage.…”

“…• Some methods rely on specific environment features such as edges of buildings [19], [20], [14], [15], [16] or semantic features [13], [21]. This limits the applicability to locations where these features are present and visible, i.e.…”