Collaborative Semantic Understanding and Mapping Framework for Autonomous Systems

Abstract

Performing collaborative semantic mapping is a critical challenge for cooperative robots to enhance their comprehensive contextual understanding of the surroundings. This paper bridges the gap between the advances in collaborative geometry mapping that relies on pure geometry information fusion, and single robot semantic mapping that focuses on integrating continuous raw sensor data. In this paper, a novel hierarchical collaborative probabilistic semantic mapping framework is proposed, where the problem is formulated in a distributed setting. The key novelty of this work is the modelling of the hierarchical semantic map fusion framework and its mathematical derivation of its probability decomposition. At the single robot level, the semantic point cloud is obtained by combining information from heterogeneous sensors and used to generate local semantic maps. At the collaborative robots level, local maps are shared among robots for global semantic map fusion. Since the voxel correspondence is unknown between local maps, an Expectation-Maximization approach is proposed to estimate the hidden data association. Then, Bayesian rule is applied to perform semantic and occupancy probability update. Experimental results on the UAV (Unmanned Aerial Vehicle) and the UGV (Unmanned Ground Vehicle) platforms show the high quality of global semantic maps, demonstrating the accuracy and utility in practical missions.

Planning the Future of Smart Cities With Swarms of Fully Autonomous Unmanned Aerial Vehicles Using a Novel Framework

“…Content may change prior to final publication. share information, and execute tasks collaboratively [119]. Therefore, FAUAVs cannot be treated as independent objects, but a part of their surroundings within SC involving the integration of human factors.…”

Self-supervised learning of LiDAR odometry based on spherical projection

“…SLOAM 21 uses semantic segmentation to detect tree trunks and the ground, separately models them, and proposes a pose optimization method based on semantic features, which achieves better results than traditional methods in a forest environment. Unlike SLOAM, Yue et al 22 proposed a semantic SLAM system that integrated camera and LiDAR. The system obtains semantic information from image data and combines the semantic information with point clouds to complete the collaborative semantic mapping.…”

The Method of Static Semantic Map Construction Based on Instance Segmentation and Dynamic Point Elimination

“…Qi [27] used YOLOv3 to obtain object types and contour to construct environmental label maps, while Guan [28] used semantic information to process point clouds and objects to construct real-time semantic maps. Yue [29] used multi-robot collaboration, and the local semantic maps are shared among robots for global semantic map fusion. Qin [30] used robust semantic features, inertial measurement unit, and wheel encoders to generate a global visual semantic map.…”