Visual Map Construction Using RGB-D Sensors for Image-Based Localization in Indoor Environments

Abstract

RGB-D sensors capture RGB images and depth images simultaneously, which makes it possible to acquire the depth information at pixel level. This paper focuses on the use of RGB-D sensors to construct a visual map which is an extended dense 3D map containing essential elements for image-based localization, such as poses of the database camera, visual features, and 3D structures of the building. Taking advantage of matched visual features and corresponding depth values, a novel local optimization algorithm is proposed to achieve point cloud registration and database camera pose estimation. Next, graph-based optimization is used to obtain the global consistency of the map. On the basis of the visual map, the image-based localization method is investigated, making use of the epipolar constraint. The performance of the visual map construction and the image-based localization are evaluated on typical indoor scenes. The simulation results show that the average position errors of the database camera and the query camera can be limited to within 0.2 meters and 0.9 meters, respectively.

Drift-Aware Monocular Localization Based on a Pre-Constructed Dense 3D Map in Indoor Environments

“…Before presenting the monocular localization approach, it is necessary to review the dense 3D map, which is called the visual map in our previous works [10]. The visual map constructed by RGB-D (RGB-depth) sensors, e.g., the Microsoft Kinect, is an extended dense 3D map that contains essential elements for vision-based localization, such as 3D point clouds, database images, and the corresponding poses of the database camera.…”

“…In the process of the visual map construction, the RGB-D sensor captures an RGB image and a depth disparity image, simultaneously. Then, the 3D point clouds are aligned using the visual map construction algorithm [10]. When the visual map construction has been completed, the 3D position of each visual feature point in indoor scenes is determined in the indoor coordinate system.…”

“…The process of visual localization is realized without any infrastructure deployment. Before the implementation of visual localization, an off-line database that contains pose-tagged database images should be created, which has been studied in existing research [9,10].…”

The Indoor Localization of a Mobile Platform Based on Monocular Vision and Coding Images

“…Localization is one of the core technologies for indoor surveying and mapping services [1,2]. In order to achieve accurate indoor navigation and positioning, many navigation and positioning technologies have been rapidly developed based on ultra wide band (UWB) [3,4], radio frequency (RF) [5,6], Wi-Fi [7,8], Bluetooth [9,10], vision [11,12], ZigBee [13,14], and multi-sensor combinations [15,16]. Vision sensors are a streaming media technology that can achieve rich texture information and transmit information in real time, which can be used to localize and collect environmental information, and have been used in the Chang'e III inspections of China and the spirit and opportunity rovers in America [17,18]).…”

“…The other category includes the works that use a stored image database annotated with the position information of the cameras, such as image fingerprinting-based methods [23,24]. Common visual positioning methods include monocular positioning [25,26], binocular positioning [27], and RGB-D positioning [12,28]. Among them, the monocular vision positioning method only needs one camera to locate a position and uses either one single frame image or stereo ISPRS Int.…”