Cross-View Policy Learning for Street Navigation

Abstract

The ability to navigate from visual observations in unfamiliar environments is a core component of intelligent agents and an ongoing challenge for Deep Reinforcement Learning (RL). Street View can be a sensible testbed for such RL agents, because it provides real-world photographic imagery at ground level, with diverse street appearances; it has been made into an interactive environment called StreetLearn [27] and used for research on navigation. However, goal-driven street navigation agents have not so far been able to transfer to unseen areas without extensive retraining, and relying on simulation is not a scalable solution. Since aerial images are easily and globally accessible, we propose instead to train a multi-modal policy on ground and aerial views, then transfer the ground view policy to unseen (target) parts of the city by utilizing aerial view observations. Our core idea is to pair the ground view with an aerial view and to learn a joint policy that is transferable across views. We achieve this by learning a similar embedding space for both views, distilling the policy across views and dropping out visual modalities. We further reformulate the transfer learning paradigm into three stages: 1) cross-modal training, when the agent is initially trained on multiple city regions, 2) aerial view-only adaptation to a new area, when the agent is adapted to a held-out region using only the easily obtainable aerial view, and 3) ground view-only transfer, when the agent is tested on navigation tasks on unseen ground views, without aerial imagery. Experimental results suggest that the proposed cross-view policy learning enables better generalization of the agent and allows for more effective transfer to unseen environments.

Deep Partial Multi-View Learning

“…Cross-View Learning (CVL) essentially searches for mappings between two views, where the similarity between the samples from different modalities can be measured directly. It has been widely applied in real applications [29], [30], [31], [32], [33], [34]. With adversarial training, the embedding spaces of two individual views are learned and aligned simultaneously [30].…”

Learning to Follow Directions in Street View

“…We implement this method as a baseline for comparison. Mirowski et al (2018) have trained deep RL agents in large-scale environments based on Google Street View and studied the transfer of navigation skills by doing limited, modular retraining in new cities, with optional adaption using aerial images (Li et al 2019).…”

CityLearn: Diverse Real-World Environments for Sample-Efficient Navigation Policy Learning

“…Our fully-configurable environment runs on top of the Unity game engine and their ML-Agents framework [68]. CityLearn is related to the recent StreetLearn work [59] used in [2], [69], [70] but has a range of useful differences. We propose the usage of diverse environments across 5 countries and additionally enable loading any other dataset including in-house recorded data; see Tables I and II for a detailed comparison. In Table I, each environment (region/dataset) also includes GPS data; except for Goald Coast Drive and Alderley Day/Nigth.…”