A Two-step Method for Extrinsic Calibration between a Sparse 3D LiDAR and a Thermal Camera

Zhang, Jun; Siritanawan, Prarinya; Yue, Yufeng; Yang, Chule; Wen, Mingxing; Wang, Danwei

doi:10.1109/icarcv.2018.8581170

Cited by 37 publications

(20 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to differences in the sensor's field of perception, only 3D LiDAR points within the camera perception area are collected. The semantic labels can be transmitted from pixels to LiDAR points by projection [24], where the parameters are calibrated by [25]. Therefore, (1) can be rewritten as:…”

Section: A Algorithm Framework and Problem Definitionmentioning

confidence: 99%

SEE-CSOM: Sharp-Edged and Efficient Continuous Semantic Occupancy Mapping for Mobile Robots

Deng

Wang

Yang

et al. 2024

IEEE Trans. Ind. Electron.

View full text Add to dashboard Cite

Generating an accurate and continuous semantic occupancy map is a key component of autonomous robotics. Most existing continuous semantic occupancy mapping methods neglect the potential differences between voxels, which reconstruct an over-inflated map. What's more, these methods have high computational complexity due to the fixed and large query range. To address the challenges of over-inflation and inefficiency, this paper proposes a novel sharp-edged and efficient continuous semantic occupancy mapping algorithm (SEE-CSOM). The main contribution of this work is to design the Redundant Voxel Filter Model (RVFM) and the Adaptive Kernel Length Model (AKLM) to improve the performance of the map. RVFM applies context entropy to filter out the redundant voxels with a low degree of confidence, so that the representation of objects will have accurate boundaries with sharp edges. AKLM adaptively adjusts the kernel length with class entropy, which reduces the amount of data used for training. Then, the multi-entropy kernel inference function is formulated to integrate the two models to generate the continuous semantic occupancy map. The algorithm has been verified on indoor and outdoor public datasets and implemented on a real robot platform, validating the significant improvement in accuracy and efficiency. The code is available at https://github.com/BIT-DYN/SEE-CSOM.

show abstract

Section: A Algorithm Framework and Problem Definitionmentioning

confidence: 99%

SEE-CSOM: Sharp-Edged and Efficient Continuous Semantic Occupancy Mapping for Mobile Robots

Deng

Wang

Yang

et al. 2024

IEEE Trans. Ind. Electron.

View full text Add to dashboard Cite

show abstract

“…Due to the limited field of view of the camera, only 3D points that fall into the semantic image will receive semantic information. Here, L i t is the 3D point in the point cloud L i t ∈ L t , T c l is the extrinsic parameter between the 3D LiDAR and the camera, K c is the intrinsic parameter of the camera, l i t is the projected point [28]. After projection, the projected point l i t is overlapped with the 2D pixel p i s t in the semantic image I s t (p i s t ∈I s t ).…”

Section: A Multimodal Semantic Information Fusionmentioning

confidence: 99%

Collaborative Semantic Understanding and Mapping Framework for Autonomous Systems

Yue

Zhao

et al. 2021

IEEE/ASME Trans. Mechatron.

Self Cite

View full text Add to dashboard Cite

Performing collaborative semantic mapping is a critical challenge for cooperative robots to enhance their comprehensive contextual understanding of the surroundings. This paper bridges the gap between the advances in collaborative geometry mapping that relies on pure geometry information fusion, and single robot semantic mapping that focuses on integrating continuous raw sensor data. In this paper, a novel hierarchical collaborative probabilistic semantic mapping framework is proposed, where the problem is formulated in a distributed setting. The key novelty of this work is the modelling of the hierarchical semantic map fusion framework and its mathematical derivation of its probability decomposition. At the single robot level, the semantic point cloud is obtained by combining information from heterogeneous sensors and used to generate local semantic maps. At the collaborative robots level, local maps are shared among robots for global semantic map fusion. Since the voxel correspondence is unknown between local maps, an Expectation-Maximization approach is proposed to estimate the hidden data association. Then, Bayesian rule is applied to perform semantic and occupancy probability update. Experimental results on the UAV (Unmanned Aerial Vehicle) and the UGV (Unmanned Ground Vehicle) platforms show the high quality of global semantic maps, demonstrating the accuracy and utility in practical missions.

show abstract

“…In their work, two sensors detect objects separately and then adopt the evidential fusion method (Dempster-Shafer theory [56]). Similarly, Zhang [57] first proposed a two-step method of calibration between a 3D LiDAR and a thermal camera. The fusion algorithms between these two sensors are not limited to low-level fusion but can be extended for high-level fusion.…”

Section: Lidar and Cameramentioning

confidence: 99%

Sensing Technology Survey for Obstacle Detection in Vegetation

Lohar

Zhu

Young

et al. 2021

Future Transportation

View full text Add to dashboard Cite

This study reviews obstacle detection technologies in vegetation for autonomous vehicles or robots. Autonomous vehicles used in agriculture and as lawn mowers face many environmental obstacles that are difficult to recognize for the vehicle sensor. This review provides information on choosing appropriate sensors to detect obstacles through vegetation, based on experiments carried out in different agricultural fields. The experimental setup from the literature consists of sensors placed in front of obstacles, including a thermal camera; red, green, blue (RGB) camera; 360° camera; light detection and ranging (LiDAR); and radar. These sensors were used either in combination or single-handedly on agricultural vehicles to detect objects hidden inside the agricultural field. The thermal camera successfully detected hidden objects, such as barrels, human mannequins, and humans, as did LiDAR in one experiment. The RGB camera and stereo camera were less efficient at detecting hidden objects compared with protruding objects. Radar detects hidden objects easily but lacks resolution. Hyperspectral sensing systems can identify and classify objects, but they consume a lot of storage. To obtain clearer and more robust data of hidden objects in vegetation and extreme weather conditions, further experiments should be performed for various climatic conditions combining active and passive sensors.

show abstract

A Two-step Method for Extrinsic Calibration between a Sparse 3D LiDAR and a Thermal Camera

Cited by 37 publications

References 13 publications

SEE-CSOM: Sharp-Edged and Efficient Continuous Semantic Occupancy Mapping for Mobile Robots

SEE-CSOM: Sharp-Edged and Efficient Continuous Semantic Occupancy Mapping for Mobile Robots

Collaborative Semantic Understanding and Mapping Framework for Autonomous Systems

Sensing Technology Survey for Obstacle Detection in Vegetation

Contact Info

Product

Resources

About