A loop closure detection method based on semantic segmentation and convolutional neural network

Li, Hongyang; Tian, Chengjun; Wang, Lequan; Lv, Hongfu

doi:10.1109/aiea53260.2021.00063

Cited by 3 publications

(3 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A deep learning-based loop closure detection algorithm is adopted in this paper, which can improve the accuracy and efficiency of loop closure detection, eliminate the accumulated error in the process of robot motion, and enhance the robustness of SLAM in complex environments. The algorithm includes unified image specification, keyframe selection, high-dimensional feature vector library and lowdimensional feature vector library construction, Etc [34].…”

Section: A Deep Learning-based Loop Closure Detection Technology Adoptedmentioning

confidence: 99%

Research on Multi-Sensor Fusion SLAM Algorithm Based on Improved Gmapping

Tian

Liu

et al. 2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

Simultaneous Localization and Mapping (SLAM) is the core technology of the intelligent robot system, and it is also the basis for its autonomous movement. In recent years, it has been found that SLAM using a single sensor has certain limitations, such as Inertial Measurement Unit (IMU) noise and serious drift, and 2D radar can only detect environmental information on the same horizontal plane. In this regard, this paper constructs a multi-sensor back-end fusion SLAM algorithm that combines vision, laser, encoder and IMU information. Experiments have proved that compared with using a single sensor, the application of a multi-sensor fusion system makes the edges of the constructed map clearer and the noise reduced. Aiming at the problem of increased calculation caused by particle degradation and too many particles, this paper improves the Gmappping algorithm, and uses the combination of selective resampling and Kullback-Leibler Distance (KLD) sampling to complete resampling. It has been proved by experiments that compared with the original algorithm of Gmapping, the application of the improved algorithm increases the particle convergence speed by 39.85% in the process of indoor mapping. Aiming at the problems that the traditional loop detection algorithm is easily affected by environmental factors, resulting in low detection accuracy, and the loop detection algorithm based on deep convolutional neural network has a large amount of calculation and takes a long time to detect. The main research of this paper is to apply a deep learning-based loop detection algorithm on the multi-sensor fusion framework, and use the combination of high-dimensional and low-dimensional features of the image for loop detection. This paper uses different algorithms to conduct comparative experiments on the dataset CityCentre. The experimental results show that compared with the traditional algorithms Bag of Words (BoW), AlexNet algorithm, VGG19 algorithm, and ResNet32 algorithm, the accuracy of the algorithm proposed in this paper has increased by 31.26%, 14.21%, 3.05%, and 1.56%, respectively. In addition, the comparison experiment results of SLAM mapping with the original Real-Time Appearance-Based Mapping (RTAB-MAP) algorithm prove that the loop closure detection algorithm based on deep learning proposed in this paper can enable the system to better build a globally consistent map, including more environmental information.

show abstract

Section: A Deep Learning-based Loop Closure Detection Technology Adoptedmentioning

confidence: 99%

Research on Multi-Sensor Fusion SLAM Algorithm Based on Improved Gmapping

Tian

Liu

et al. 2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the encoder-decoder structure, precision and computation time can be balanced by dilated convolution [30]. In reference [31], Li et al used semantic segmentation to remove dynamic targets, retained the background part of the image, and then combined with CNN features of different dimensions for loop closure detection. is method eliminates interference by using semantic information and avoids the influence of dynamic targets on loop closure detection.…”

Section: Related Workmentioning

confidence: 99%

Loop Closure Detection Based on Image Semantic Segmentation in Indoor Environment

Wang

et al. 2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

When mobile robots run in indoor environment, a large number of similar images are easy to appear in the images collected, probably causing false-positive judgment in loop closure detection based on simultaneous localization and mapping (SLAM). To solve this problem, a loop closure detection algorithm for visual SLAM based on image semantic segmentation is proposed in this paper. Specifically, the current frame is semantically segmented by optimized DeepLabv3+ model to obtain semantic labels in the image. The 3D semantic node coordinates corresponding to each semantic label are then extracted by combining mask centroid and image depth information. According to the distribution of semantic nodes, the DBSCAN density clustering algorithm is adopted to cluster densely distributed semantic nodes to avoid mismatching due to the close distance of semantic nodes in the subsequent matching process. Finally, the multidimensional similarity comparison of first rough and then fine is adopted to screen the candidate frames of loop closure from key frames and then confirm the real loop closure to complete accurate loop closure detection. Testing with public datasets and self-filmed datasets, experimental results show that being well adapted to illumination change, viewpoint deviation, and item movement or missing, the proposed algorithm can effectively improve the accuracy of loop closure detection in indoor environment.

show abstract

“…Semantic segmentation has gradually been applied in the field of loop closure detection in recent years. For example, in [21], Li et al used semantic information to exclude dynamic target interference for loop closure detection, obtaining high-and low-dimensional convolutional neural network (CNN) features of images, and combining CNN features of different dimensions for loop closure detection. In [22], Wu et al applied semantic segmentation to extract semantic information of images before calibrating the convolutional features acquired by a CNN using the semantic features, constructed the descriptors (TNNLoST), and finally, completed the loop closure detection based on the TNNLoST.…”

Section: Introductionmentioning

confidence: 99%

Loop Closure Detection for Mobile Robot based on Multidimensional Image Feature Fusion

Wang

et al. 2022

Machines

View full text Add to dashboard Cite

Loop closure detection is a crucial part of VSLAM. However, the traditional loop closure detection algorithms are difficult to adapt to complex and changeable scenes. In this paper, we fuse Gist features, semantic features and appearance features of the image to detect the loop closures quickly and accurately. Firstly, we take advantage of the fast extraction speed of the Gist feature by using it to screen the loop closure candidate frames. Then, the current frame and the candidate frame are semantically segmented to obtain the mask blocks of various types of objects, and the semantic nodes are constructed to calculate the semantic similarity between them. Next, the appearance similarity between the images is calculated according to the shape of the mask blocks. Finally, based on Gist similarity, semantic similarity and appearance similarity, the image similarity calculation model can be built as the basis for loop closure detection. Experiments are carried out on both public and self-filmed datasets. The results show that our proposed algorithm can detect the loop closure in the scene quickly and accurately when the illumination, viewpoint and object change.

show abstract

A loop closure detection method based on semantic segmentation and convolutional neural network

Cited by 3 publications

References 4 publications

Research on Multi-Sensor Fusion SLAM Algorithm Based on Improved Gmapping

Research on Multi-Sensor Fusion SLAM Algorithm Based on Improved Gmapping

Loop Closure Detection Based on Image Semantic Segmentation in Indoor Environment

Loop Closure Detection for Mobile Robot based on Multidimensional Image Feature Fusion

Contact Info

Product

Resources

About