Towards a Meaningful 3D Map Using a 3D Lidar and a Camera

Jeong, Jongmin; Yoon, Tae Sung; Park, Jin Bae

doi:10.3390/s18082571

Cited by 27 publications

(14 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wang and Kim [34] use images and 3D point clouds from the KITTI dataset [10] to jointly estimate road layout and segment urban scenes semantically by applying a relative location prior. Jeong et al [11], [12] also propose a multi-modal sensor-based semantic 3D mapping system to improve the segmentation results in terms of the intersection-over-union (IoU) metric, in large-scale environments as well as in environments with few features. Liang et al [16] propose a novel 3D object detector that can exploit both LiDAR and camera data to perform accurate object localization.…”

Section: Introductionmentioning

confidence: 99%

SuMa++: Efficient LiDAR-based Semantic SLAM

Chen

Milioto

Palazzolo

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

399

192

View full text Add to dashboard Cite

Reliable and accurate localization and mapping are key components of most autonomous systems. Besides geometric information about the mapped environment, the semantics plays an important role to enable intelligent navigation behaviors. In most realistic environments, this task is particularly complicated due to dynamics caused by moving objects, which can corrupt the mapping step or derail localization. In this paper, we propose an extension of a recently published surfelbased mapping approach exploiting three-dimensional laser range scans by integrating semantic information to facilitate the mapping process. The semantic information is efficiently extracted by a fully convolutional neural network and rendered on a spherical projection of the laser range data. This computed semantic segmentation results in point-wise labels for the whole scan, allowing us to build a semantically-enriched map with labeled surfels. This semantic map enables us to reliably filter moving objects, but also improve the projective scan matching via semantic constraints. Our experimental evaluation on challenging highways sequences from KITTI dataset with very few static structures and a large amount of moving cars shows the advantage of our semantic SLAM approach in comparison to a purely geometric, state-of-the-art approach.

show abstract

Section: Introductionmentioning

confidence: 99%

SuMa++: Efficient LiDAR-based Semantic SLAM

Chen

Milioto

Palazzolo

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

399

192

View full text Add to dashboard Cite

show abstract

“…In this scenario, a fleet of vehicles could all contribute to the same exploration, requiring collaborative behaviors for coordination and information sharing between them [39] or even to offload the most computationally intensive tasks to an external infrastructure (cloud robotics) [40]. Lastly, possible combining the RGB data, the robot could detect a new object placed before it, to then classify it [41]. This classification would be useful for different tasks, for example, choosing the proper gripper configuration to grab it or to place it in the desired container, if we think of a task where the goal of the robot is to separate the objects by categories.…”

Section: Resultsmentioning

confidence: 99%

Autonomous Scene Exploration for Robotics: A Conditional Random View-Sampling and Evaluation Using a Voxel-Sorting Mechanism for Efficient Ray Casting

Santos

Oliveira

Arrais

et al. 2020

Sensors

View full text Add to dashboard Cite

Carrying out the task of the exploration of a scene by an autonomous robot entails a set of complex skills, such as the ability to create and update a representation of the scene, the knowledge of the regions of the scene which are yet unexplored, the ability to estimate the most efficient point of view from the perspective of an explorer agent and, finally, the ability to physically move the system to the selected Next Best View (NBV). This paper proposes an autonomous exploration system that makes use of a dual OcTree representation to encode the regions in the scene which are occupied, free, and unknown. The NBV is estimated through a discrete approach that samples and evaluates a set of view hypotheses that are created by a conditioned random process which ensures that the views have some chance of adding novel information to the scene. The algorithm uses ray-casting defined according to the characteristics of the RGB-D sensor, and a mechanism that sorts the voxels to be tested in a way that considerably speeds up the assessment. The sampled view that is estimated to provide the largest amount of novel information is selected, and the system moves to that location, where a new exploration step begins. The exploration session is terminated when there are no more unknown regions in the scene or when those that exist cannot be observed by the system. The experimental setup consisted of a robotic manipulator with an RGB-D sensor assembled on its end-effector, all managed by a Robot Operating System (ROS) based architecture. The manipulator provides movement, while the sensor collects information about the scene. Experimental results span over three test scenarios designed to evaluate the performance of the proposed system. In particular, the exploration performance of the proposed system is compared against that of human subjects. Results show that the proposed approach is able to carry out the exploration of a scene, even when it starts from scratch, building up knowledge as the exploration progresses. Furthermore, in these experiments, the system was able to complete the exploration of the scene in less time when compared to human subjects.

show abstract

“…With the recent developments in LiDAR technology, numerous mapping systems have been proposed in the SLAM community [ 38 , 39 , 40 , 41 , 42 , 43 , 44 ]. Although LiDARs are precise and less noisy than RGB-D sensors, it remains challenging to build efficient 3D maps with such sensors.…”

Section: Related Workmentioning

confidence: 99%

VDBFusion: Flexible and Efficient TSDF Integration of Range Sensor Data

Vizzo

Guadagnino

Behley

et al. 2022

Sensors

View full text Add to dashboard Cite

Mapping is a crucial task in robotics and a fundamental building block of most mobile systems deployed in the real world. Robots use different environment representations depending on their task and sensor setup. This paper showcases a practical approach to volumetric surface reconstruction based on truncated signed distance functions, also called TSDFs. We revisit the basics of this mapping technique and offer an approach for building effective and efficient real-world mapping systems. In contrast to most state-of-the-art SLAM and mapping approaches, we are making no assumptions on the size of the environment nor the employed range sensor. Unlike most other approaches, we introduce an effective system that works in multiple domains using different sensors. To achieve this, we build upon the Academy-Award-winning OpenVDB library used in filmmaking to realize an effective 3D map representation. Based on this, our proposed system is flexible and highly effective and, in the end, capable of integrating point clouds from a 64-beam LiDAR sensor at 20 frames per second using a single-core CPU. Along with this publication comes an easy-to-use C++ and Python library to quickly and efficiently solve volumetric mapping problems with TSDFs.

show abstract

Towards a Meaningful 3D Map Using a 3D Lidar and a Camera

Cited by 27 publications

References 32 publications

SuMa++: Efficient LiDAR-based Semantic SLAM

SuMa++: Efficient LiDAR-based Semantic SLAM

Autonomous Scene Exploration for Robotics: A Conditional Random View-Sampling and Evaluation Using a Voxel-Sorting Mechanism for Efficient Ray Casting

VDBFusion: Flexible and Efficient TSDF Integration of Range Sensor Data

Contact Info

Product

Resources

About