Look No Deeper: Recognizing Places from Opposing Viewpoints under Varying Scene Appearance using Single-View Depth Estimation

Abstract

Visual place recognition (VPR) -the act of recognizing a familiar visual place -becomes difficult when there is extreme environmental appearance change or viewpoint change. Particularly challenging is the scenario where both phenomena occur simultaneously, such as when returning for the first time along a road at night that was previously traversed during the day in the opposite direction. While such problems can be solved with panoramic sensors, humans solve this problem regularly with limited field of view vision and without needing to constantly turn around. In this paper, we present a new depth-and temporal-aware visual place recognition system that solves the opposing viewpoint, extreme appearance-change visual place recognition problem. Our system performs sequence-to-single matching by extracting depth-filtered keypoints using a stateof-the-art depth estimation pipeline, constructing a keypoint sequence over multiple frames from the reference dataset, and comparing those keypoints to those in a single query image. We evaluate the system on a challenging benchmark dataset and show that it consistently outperforms state-of-theart techniques. We also develop a range of diagnostic simulation experiments that characterize the contribution of depth-filtered keypoint sequences with respect to key domain parameters including degree of appearance change and camera motion.

Mapillary Street-Level Sequences: A Dataset for Lifelong Place Recognition

“…Radenovic et al [42] proposes a pipeline to obtain large 3D scene reconstructions from unordered images and uses these 3D reconstructions as ground truth for training a Generalized Mean (GeM) layer with hard positive and negative mining. Garg et al [18], on the other hand, uses single-view depth predictions to recognize places revisited from opposite directions. Also, addressing extreme viewpoint changes, Garg et al [19] suggests semantically aggregating salient visual information.…”

Place recognition survey: An update on deep learning approaches

“…To this end, place recognition approaches rely on various mapping frameworks and map update mechanisms. Regarding the mapping frameworks, three main approaches are highlighted: database [26], [27], topological [28]- [30] or topologicalmetric [31], [32].…”

“…Hierarchical methods combine multiple matching approaches in a single place recognition framework. For example, the coarse-to-fine architecture [27], [52] selects top candidates in a coarse tier, and from those, selects the best match in a fine tier.…”

Unsupervised Monocular Depth Estimation for Night-Time Images Using Adversarial Domain Feature Adaptation

“…Estimating depth from RGB images is a challenging problem which finds applications in a wide range of fields such as augmented reality [30], 3D reconstruction [16], self-driving cars [19], place recognition [11], etc. The recent success of deep learning methods has spurred the research in this field leading to the creation of several new benchmarks that now outperform traditional methods which rely on handcrafted features and exploit camera geometry and/or camera motion for depth and pose estimation from monocular or stereo sequence of images (video).…”

“…The modular encoder-decoder architecture provides the flexibility of using the encoder module as a feature extractor to extract or select useful features from input images. Such feature extractors are used in several applications such as pose estimation [18], Visual Place Recognition (VPR) [12] [11], object detection [41] and segmentation [6]. We demonstrate one such application where the adapted features obtained from our encoder module are shown to provide superior place recognition accuracy compared to other state-of-the-art feature representations available in the literature.…”