Real-time RGB-D semantic keyframe SLAM based on image segmentation learning from industrial CAD models

Abstract

This paper presents methods for performing realtime semantic SLAM aimed at autonomous navigation and control of a humanoid robot in a manufacturing scenario. A novel multi-keyframe approach is proposed that simultaneously minimizes a semantic cost based on class-level features in addition to common photometric and geometric costs. The approach is shown to robustly construct a 3D map with associated class labels relevant to robotic tasks. Alternatively to existing approaches, the segmentation of these semantic classes have been learnt using RGB-D sensor data aligned with an industrial CAD manufacturing model to obtain noisy pixel-wise labels. This dataset confronts the proposed approach in a complicated real-world setting and provides insight into the practical use case scenarios. The semantic segmentation network was fine tuned for the given use case and was trained in a semisupervised manner using noisy labels. The developed software is real-time and integrated with ROS to obtain a complete semantic reconstruction for the control and navigation of the HRP4 robot. Experiments in-situ at the Airbus manufacturing site in Saint-Nazaire validate the proposed approach.

Autonomous Intersection Management: Optimal Trajectories and Efficient Scheduling

“…Despite the advances in the field of positioning systems, obtaining accurate positions remains a very active research area. In addition to GPS-RTK (Global Positioning System-Real-Time Kinematic) [ 120 ], SLAM (Simultaneous Localization And Mapping) [ 121 ], and inertial units [ 122 ], the authors propose ground position markers [ 17 ] or cameras that observe the scene to help CAVs [ 118 ]. Currently, new technologies of positioning allow foreseeing position accuracy unattainable in the past [ 123 ].…”

Humanoid Robots in Aircraft Manufacturing: The Airbus Use Cases

“…Semantic information can also be fed back to the SLAM. The measurement vector M is extended to simultaneously minimise photometric, geometric and semantic costs in realtime [18]. The approach has been shown to robustly construct a labeled large-scale 3D map relevant to robotic tasks.…”

Enhancing Semantic Segmentation through Reinforced Active Learning: Combating Dataset Imbalances and Bolstering Annotation Efficiency

“…These efficient DNNs are characterized by their low computational demands and quick inference times [12] , and their widespread adoption has significantly influenced applications in various fields such as autonomous driving [13,14] , semantic segmentation enables precise scene understanding, allowing the vehicle to identify and differentiate between various objects on the road, such as pedestrians, vehicles, traffic signs, and obstacles. This technology aids in real-time decision-making, helping the vehicle navigate complex environments and ensure the safety of passengers and pedestrians; robot manipulation [15,16] , for robots to interact intelligently with their environment, they require a comprehensive understanding of the objects and structures in their surroundings. Semantic segmentation facilitates this by enabling robots to identify and differentiate between different objects and their corresponding spatial relationships.…”