Indoor Topological Localization Using a Visual Landmark Sequence

Abstract

This paper presents a novel indoor topological localization method based on mobile phone videos. Conventional methods suffer from indoor dynamic environmental changes and scene ambiguity. The proposed Visual Landmark Sequence-based Indoor Localization (VLSIL) method is capable of addressing problems by taking steady indoor objects as landmarks. Unlike many feature or appearance matching-based localization methods, our method utilizes highly abstracted landmark sematic information to represent locations and thus is invariant to illumination changes, temporal variations, and occlusions. We match consistently detected landmarks against the topological map based on the occurrence order in the videos. The proposed approach contains two components: a convolutional neural network (CNN)-based landmark detector and a topological matching algorithm. The proposed detector is capable of reliably and accurately detecting landmarks. The other part is the matching algorithm built on the second order hidden Markov model and it can successfully handle the environmental ambiguity by fusing sematic and connectivity information of landmarks. To evaluate the method, we conduct extensive experiments on the real world dataset collected in two indoor environments, and the results show that our deep neural network-based indoor landmark detector accurately detects all landmarks and is expected to be utilized in similar environments without retraining and that VLSIL can effectively localize indoor landmarks.

Geometric property-based convolutional neural network for indoor object detection

“…17 Zhu et al proposed a CNN-based indoor landmark detector with the help of a topological matching algorithm. 18 Together with classical classifier, Jiang et al proposed a CNN-based tracking method for person-following robot. 19 Loghmani et al proposed a two-stream fusion method for robot vision, 20 in which the features of the two CNN streams of RGB and depth images are fed into an RNN to detect objects.…”

Forward and Backward Visual Fusion Approach to Motion Estimation with High Robustness and Low Cost

“…In addition, ego-motion estimation in vehicles and robots is fundamental as it is usually the pre-requisite for higher-layer tasks, such as robot-based surveillance, autonomous navigation, path planning, for example, References [6,7]. A vision-based odometry system, compared to a traditional wheel-based or satellites-based localization system, has the advantages of an impervious character to inherent sensor inefficacies [8,9] (e.g., wheel encoder error because of uneven, slippery terrain or other adverse conditions) and can be used in a GPS-denied area [10,11] (e.g., underwater and tunnels in urban environments.) The proposed approach utilizes only visual perception cameras with lightweight, high robustness and low-cost characters.…”

Uncertainty-based localization in a topological robot navigation system

“…Robot localization, including topological approaches, has been widely researched by the robotics community for the last decades. According to [139], there are three types of approaches to the topological localization problem: structure-based methods that represents the environment as a topological map and predicts robot pose by finding the sequence of observations and transitions that best matches the structure of the map [1,11,35,41,144,168,184]; image-based methods that model the environment as a database of images linked to locations [5,7,20,29,158,166]. They use image retrieval techniques to identify the database images most relevant to the query, which are then used to estimate the pose of the robot; and, learning-based methods that represent the scene by a learned model, which either predicts matches for pose estimation or directly regresses the pose [6,107,123].…”

“…The comparison is performed by a geometric descriptor of the GVD meet points based on emanating edges and their angles and, distance to obstacles. In [184], Zhu et al exploit the landmark sequences of steady objects using a second order HMM. Common objects are used as landmarks and the occurrence order allow to match the current location of the robot to the topological map.…”