Visual SLAM Integration With Semantic Segmentation and Deep Learning: A Review

Pu, Huayan; Luo, Jun; Wang, Gang; Huang, Tao; Liu, Hongliang; Luo, Jun

doi:10.1109/jsen.2023.3306371

Cited by 33 publications

(2 citation statements)

References 138 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Visual odometry and visual SLAM for mobile robots 2015 [16] Multi-robot SLAM systems 2016 [17] Vision SLAM Algorithms 2017 [18] RGB-D SLAM 2018 [19] Deep Learning for Visual SLAM in Transportation Robotics 2019 [20] Semantic SLAM 2020 [21] Solid State LiDAR(SSL) 2021 [22] V-SLAM 2022 [23] Deep Learning and Semantic Segmentation combined with Visual SLAM 2023 [24] 2.1.…”

Section: Thematic Year Resultsmentioning

confidence: 99%

Path Planning Technique for Mobile Robots: A Review

Yang,

Li,

Qian

et al. 2023

Machines

View full text Add to dashboard Cite

Mobile robot path planning involves designing optimal routes from starting points to destinations within specific environmental conditions. Even though there are well-established autonomous navigation solutions, it is worth noting that comprehensive, systematically differentiated examinations of the critical technologies underpinning both single-robot and multi-robot path planning are notably scarce. These technologies encompass aspects such as environmental modeling, criteria for evaluating path quality, the techniques employed in path planning and so on. This paper presents a thorough exploration of techniques within the realm of mobile robot path planning. Initially, we provide an overview of eight diverse methods for mapping, each mirroring the varying levels of abstraction that robots employ to interpret their surroundings. Furthermore, we furnish open-source map datasets suited for both Single-Agent Path Planning (SAPF) and Multi-Agent Path Planning (MAPF) scenarios, accompanied by an analysis of prevalent evaluation metrics for path planning. Subsequently, focusing on the distinctive features of SAPF algorithms, we categorize them into three classes: classical algorithms, intelligent optimization algorithms, and artificial intelligence algorithms. Within the classical algorithms category, we introduce graph search algorithms, random sampling algorithms, and potential field algorithms. In the intelligent optimization algorithms domain, we introduce ant colony optimization, particle swarm optimization, and genetic algorithms. Within the domain of artificial intelligence algorithms, we discuss neural network algorithms and fuzzy logic algorithms. Following this, we delve into the different approaches to MAPF planning, examining centralized planning which emphasizes decoupling conflicts, and distributed planning which prioritizes task execution. Based on these categorizations, we comprehensively compare the characteristics and applicability of both SAPF and MAPF algorithms, while highlighting the challenges that this field is currently grappling with.

show abstract

Section: Thematic Year Resultsmentioning

confidence: 99%

Path Planning Technique for Mobile Robots: A Review

Yang,

Li,

Qian

et al. 2023

Machines

View full text Add to dashboard Cite

show abstract

“…In recent years, with the thriving development of deep learning, target detection, and semantic segmentation have found extensive applications in the field of image processing. Some scholars have started integrating these image processing techniques based on deep learning with traditional SLAM methods to enhance the system's robustness and accuracy in dynamic environments [23,24].…”

Section: Slam Combines Deep Learningmentioning

confidence: 99%

SLM-SLAM: a visual SLAM system based on segmented large-scale model in dynamic scenes and zero-shot conditions

Zhu,

Chen,

Jiang

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

In the real world, the presence of various dynamic objects can impact the localization accuracy of the majority of typical Visual Simultaneous Localization and Mapping (VSLAM) systems. Simultaneously, many dynamic VSLAM systems based on neural networks require pre-training for specific application scenarios. We present SLM-SLAM, the first VSLAM system that implements zero-shot processing of dynamic scenes. It achieves the capability to handle various dynamic objects without the necessity for pre-training, enabling straightforward adaptation to different application scenarios. Firstly, we designed an open-world semantic segmentation module based on a segmented large-scale model to acquire semantic information in the scene. Subsequently, we devised a label-based strategy for selecting feature points, jointly optimizing poses with the weighted labels provided by both semantic and geometric information. Finally, we refined the keyframe selection strategy of ORB-SLAM3 to prevent matching errors caused by an insufficient number of remaining static feature points in the scene. We conducted experiments on the TUM dataset, the KITTI dataset, and real world scenarios. The results indicate that in dynamic scenes, our SLM-SLAM significantly improves localization accuracy compared to ORB-SLAM3, and its performance is comparable to state-of-the-art dynamic VSLAM systems.

show abstract

Visual Inertial SLAM Based on Spatiotemporal Consistency Optimization in Diverse Environments

Pu,

Luo,

Wang

et al. 2024

Journal of Field Robotics

View full text Add to dashboard Cite

Currently, the majority of robots equipped with visual‐based simultaneous mapping and localization (SLAM) systems exhibit good performance in static environments. However, practical scenarios often present dynamic objects, rendering the environment less than entirely “static.” Diverse dynamic objects within the environment pose substantial challenges to the precision of visual SLAM system. To address this challenge, we propose a real‐time visual inertial SLAM system that extensively leverages objects within the environment. First, we reject regions corresponding to dynamic objects. Following this, geometric constraints are applied within the stationary object regions to elaborate the mask of static areas, thereby facilitating the extraction of more stable feature points. Second, static landmarks are constructed based on the static regions. A spatiotemporal factor graph is then created by combining the temporal information from the Inertial Measurement Unit (IMU) with the semantic information from the static landmarks. Finally, we perform a diverse set of validation experiments on the proposed system, encompassing challenging scenarios from publicly available benchmarks and the real world. Within these experimental scenarios, we compare with state‐of‐the‐art approaches. More specifically, our system achieved a more than 40% accuracy improvement over baseline method in these data sets. The results demonstrate that our proposed method exhibits outstanding robustness and accuracy not only in complex dynamic environments but also in static environments.

show abstract

Visual SLAM Integration With Semantic Segmentation and Deep Learning: A Review

Cited by 33 publications

References 138 publications

Path Planning Technique for Mobile Robots: A Review

Path Planning Technique for Mobile Robots: A Review

SLM-SLAM: a visual SLAM system based on segmented large-scale model in dynamic scenes and zero-shot conditions

Visual Inertial SLAM Based on Spatiotemporal Consistency Optimization in Diverse Environments

Contact Info

Product

Resources

About