Robust and Efficient Indoor Localization Using Sparse Semantic Information from a Spherical Camera

Abstract

Self-localization enables a system to navigate and interact with its environment. In this study, we propose a novel sparse semantic self-localization approach for robust and efficient indoor localization. “Sparse semantic” refers to the detection of sparsely distributed objects such as doors and windows. We use sparse semantic information to self-localize on a human-readable 2D annotated map in the sensor model. Thus, compared to previous works using point clouds or other dense and large data structures, our work uses a small amount of sparse semantic information, which efficiently reduces uncertainty in real-time localization. Unlike complex 3D constructions, the annotated map required by our method can be easily prepared by marking the approximate centers of the annotated objects on a 2D map. Our approach is robust to the partial obstruction of views and geometrical errors on the map. The localization is performed using low-cost lightweight sensors, an inertial measurement unit and a spherical camera. We conducted experiments to show the feasibility and robustness of our approach.

All-Around 3D Reconstruction from Spherical Images with Semantic Segmentation

“…A difference between our approach and the methods mentioned above is that our work relies on semantic information. Semantic classification of spherical images has already been utilized in some localization and mapping applications, such as the 2D indoor localization method in [7], which is based on the identification of sparsely distributed objects. Another idea relying on lower-level semantic categories is presented in [8], where the geometric context of indoor structures is inferred from location and color data.…”

A Spatial-Frequency Domain Associated Image-Optimization Method for Illumination-Robust Image Matching

“…The technique possesses the merits of high computational efficiency, high theoretical accuracy, and insensitivity to geometric deformations and differences [ 7 ]. As a result, feature-based image matching has received a lot of attention in the field of computer vision [ 8 , 9 , 10 , 11 ], photogrammetry and remote sensing [ 12 , 13 , 14 , 15 ], in applications [ 16 , 17 , 18 ] such as multiple view 3D reconstruction, remote sensing image fusion and visual localization.…”

Real-Time Plane Detection with Consistency from Point Cloud Sequences

“…Thus, planar structure recognition, which can be formulated as the plane detection problem, has become an important research topic in computer vision for decades. The detected planes, which can be regarded as the abstracted form of an actual scene, contain a lot of high-level structure information and they can benefit many other semantic analysis tasks, like object detection [ 1 ], self-navigation [ 2 ], scene segmentation [ 3 ], SLAM [ 4 , 5 ], robot self-localization [ 6 , 7 , 8 ], For instance, the robot can better map the current environment with the plane detection result, which significantly reduces the uncertainty in the mapping results and improves the accuracy of positioning.…”