Integrating Dense LiDAR-Camera Road Detection Maps by a Multi-Modal CRF Model

Gu, Shuo; Zhang, Yigong; Tang, Jinhui; Yang, Jian; Álvarez, Jose M.; Kong, Hui

doi:10.1109/tvt.2019.2946100

Cited by 24 publications

(14 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[1] focuses on 2D object detection, it combines the proposals from two branches along with features like confidence score, and the model outputs the final IoU score. [29], [28] solves the road detection by combing the segmentation results together. As late-fusion in [53], it summarizes the scores from different branches for the same 3D detection proposal into one final score.…”

Section: Images Semantic Segmentationmentioning

confidence: 99%

Multi-modal Sensor Fusion for Auto Driving Perception: A Survey

Huang¹,

Shi²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Multi-modal fusion is a fundamental task for the perception of an autonomous driving system, which has recently intrigued many researchers. However, achieving a rather good performance is not an easy task due to the noisy raw data, underutilized information, and the misalignment of multi-modal sensors. In this paper, we provide a literature review of the existing multi-modal-based methods for perception tasks in autonomous driving. Generally, we make a detailed analysis including over 50 papers leveraging perception sensors including LiDAR and camera trying to solve object detection and semantic segmentation tasks. Different from traditional fusion methodology for categorizing fusion models, we propose an innovative way that divides them into two major classes, four minor classes by a more reasonable taxonomy in the view of the fusion stage. Moreover, we dive deep into the current fusion methods, focusing on the remaining problems and open-up discussions on the potential research opportunities. In conclusion, what we expect to do in this paper is to present a new taxonomy of multi-modal fusion methods for the autonomous driving perception tasks and provoke thoughts of the fusion-based techniques in the future.

show abstract

Section: Images Semantic Segmentationmentioning

confidence: 99%

Multi-modal Sensor Fusion for Auto Driving Perception: A Survey

Huang¹,

Shi²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The energy contained 2D unary potential, 3D potential, and 2D-3D pairwise potential. Based on conference [25], this author further considered the distribution of projection points and proposed an improved Delaunay triangular upsampling strategy [26].…”

Section: Multiple-sensor Fusion Based Road Detectionmentioning

confidence: 99%

“…Multi-modality sensor fusion [10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26] has been a way to improve the perception robustness in autonomous vehicles.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Lidar-Camera Fusion For Road Detection Using Recurrent Conditional Random Field Model

Wang

2022

Preprint

View full text Add to dashboard Cite

Reliable road detection is an essential task in autonomous driving systems. Two categories of sensors are commonly used: camera and LiDAR, each of which can provide corresponding supplemental. Nevertheless, existing sensor fusion methods do not fully utilize multimodal data. Most of them are dominated by image or taking point cloud as a supplement rather than making the best of them, the correlation between modalities is ignored. This paper proposes Recurrent Conditional Random Field Model (R-CRF) for road detection by fusing image and point cloud segmentation. The R-CRF integrates results (information) from modalities in a probabilistic way. Each modality is independently processed with its own semantic segmentation network. The probability scores obtained are considered as unary term for individual pixel nodes in random field while RGB image and the densified LiDAR-images are used as pairwise terms. The energy function is then iteratively optimized by mean-field variational inference so that we can refine the labeling results by exploiting fully-connected graphs of the RGB image and LiDAR-images. Extensive experiments are conducted on KITTI-Road dataset, and these demonstrate the method achieves competitive performance.

show abstract

“…Multimodality sensor fusion 8 – 23 has been used to improve the perception robustness in autonomous vehicles. A camera provides texture and colours, but the nature of the passive sensor makes it susceptible to variations in environmental lighting.…”

Section: Introductionmentioning

confidence: 99%

LiDAR–camera fusion for road detection using a recurrent conditional random field model

Wang

2022

Sci Rep

View full text Add to dashboard Cite

Reliable road detection is an essential task in autonomous driving systems. Two categories of sensors are commonly used, cameras and light detection and ranging (LiDAR), each of which can provide corresponding supplements. Nevertheless, existing sensor fusion methods do not fully utilize multimodal data. Most of them are dominated by images and take point clouds as a supplement rather than making the best of them, and the correlation between modalities is ignored. This paper proposes a recurrent conditional random field (R-CRF) model to fuse images and point clouds for road detection. The R-CRF model integrates results (information) from modalities in a probabilistic way. Each modality is independently processed with its semantic segmentation network. The probability scores obtained are considered a unary term for individual pixel nodes in a random field, while RGB images and the densified LiDAR images are used as pairwise terms. The energy function is then iteratively optimized by mean-field variational inference, and the labelling results are refined by exploiting fully connected graphs of the RGB image and LiDAR images. Extensive experiments are conducted on the public KITTI-Road dataset, and the proposed method achieves competitive performance.

show abstract

Integrating Dense LiDAR-Camera Road Detection Maps by a Multi-Modal CRF Model

Cited by 24 publications

References 33 publications

Multi-modal Sensor Fusion for Auto Driving Perception: A Survey

Multi-modal Sensor Fusion for Auto Driving Perception: A Survey

Lidar-Camera Fusion For Road Detection Using Recurrent Conditional Random Field Model

LiDAR–camera fusion for road detection using a recurrent conditional random field model

Contact Info

Product

Resources

About