Hybrid conditional random field based camera-LIDAR fusion for road detection

Xiao, Liang; Wang, Ruili; Dai, Bin; Fang, Yuqiang; Liu, Daxue; Wu, Tao

doi:10.1016/j.ins.2017.04.048

Cited by 145 publications

(75 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar image view was employed by Han et al [16] and Gu et al [15] followed by feature extraction using histogram. Gonzalez et al [14] and Xiao et al [38] created a dense depth map from point cloud and then combined the map with the camera data for their machine learning based road boundary detector. Similarly, the multi-view method [6] transformed point cloud into both image and top views and then combined with camera data for sensor fusion using a CNN.…”

Section: Related Workmentioning

confidence: 99%

ChipNet: Real-Time LiDAR Processing for Drivable Region Segmentation on an FPGA

Lyu

Bai

Huang

2019

IEEE Trans. Circuits Syst. I

View full text Add to dashboard Cite

This paper presents a field-programmable gate array (FPGA) design of a segmentation algorithm based on convolutional neural network (CNN) that can process light detection and ranging (LiDAR) data in real-time. For autonomous vehicles, drivable region segmentation is an essential step that sets up the static constraints for planning tasks. Traditional drivable region segmentation algorithms are mostly developed on camera data, so their performance is susceptible to the light conditions and the qualities of road markings. LiDAR sensors can obtain the 3D geometry information of the vehicle surroundings with high precision. However, it is a computational challenge to process a large amount of LiDAR data in real-time. In this paper, a convolutional neural network model is proposed and trained to perform semantic segmentation using data from the LiDAR sensor. An efficient hardware architecture is proposed and implemented on an FPGA that can process each LiDAR scan in 17.59 ms, which is much faster than the previous works. Evaluated using Ford and KITTI road detection benchmarks, the proposed solution achieves both high accuracy in performance and real-time processing in speed.

show abstract

Section: Related Workmentioning

confidence: 99%

ChipNet: Real-Time LiDAR Processing for Drivable Region Segmentation on an FPGA

Lyu

Bai

Huang

2019

IEEE Trans. Circuits Syst. I

View full text Add to dashboard Cite

show abstract

“…In the Lidar-histogram representation, the 3D traversable road plane in front of vehicle can be projected as a straight line, and the positive and negative obstacles are projected above and below the line respectively. Liang Xiao et al proposed a hybrid CRF to fuse Lidar and image data [23]. They firstly extracted features from images and used an boosted decision tree classifier to predict the unary potential.…”

Section: Related Workmentioning

confidence: 99%

Road segmentation with image-LiDAR data fusion in deep neural network

Liu

Yao

Sun

et al. 2019

Multimed Tools Appl

View full text Add to dashboard Cite

Robust road segmentation is a key challenge in self-driving research. Though many image based methods have been studied and high performances in dataset evaluations have been reported, developing robust and reliable road segmentation is still a major challenge. Data fusion across different sensors to improve the performance of road segmentation is widely considered an important and irreplaceable solution.In this paper, we propose a novel structure to fuse image and LiDAR point cloud in an end-to-end semantic segmentation network, in which the fusion is performed at decoder stage instead of at, more commonly, encoder stage. During fusion, we improve the multi-scale LiDAR map generation to increase the precision of multi-scale LiDAR map by introducing pyramid projection method. Additionally, we adapted the multi-path refinement network with our fusion strategy and improve the road prediction compared with transpose convolution with skip layers. Our approach has been tested on KITTI ROAD dataset and have a competitive performance.

show abstract

“…Kühnl et al 19 proposed an approach based on the computation of spatial ray features which generates many rays in several directions of each base point and applies trained classifier on the accumulated ray features to detect the lanes and road. Xiao et al [20][21][22] use boosted trees to train the road model and fuse the color and Lidar information into the conditional random field to obtain a coincident result. Deep learning-based methods 23 have achieved nearly a perfect result that relies on powerful deep network architectures which shows that for certain scenes, the models can be well learned but it is not sure that the models can be transferred in other unseen environment without retraining.…”

Section: Related Workmentioning

confidence: 99%

Gaussian process regression-based robust free space detection for autonomous vehicle by 3-D point cloud and 2-D appearance information fusion

Xiao

Dai

et al. 2017

International Journal of Advanced Robotic Systems

Self Cite

View full text Add to dashboard Cite

Free space detection is crucial to autonomous vehicles while existing works are not entirely satisfactory. As cameras have many advantages on environment perception, a stereo vision-based robust free space detection method is proposed which mainly depends on geometry information and Gaussian process regression. In this work, in order to improve the performance by exploiting multiple source information, we apply Bayesian framework and conditional random field inference to fuse the multimodal information including 2-D image and 3-D point geometric information. Particularly, a Bayesian framework is used for multiple feature fusion to provide a normalized and flexible output. Gaussian process regression is used to automatically and incrementally regress the data, resulting enhanced performance. Finally, conditional random field with color and geometry constrains is applied to make the result more robust. In order to evaluate the proposed method, quantitative experiments on popular KITTI-road data set and qualitative experiments on our own campus data set are tested. The results show satisfactory and inspiring performance compared to the outstanding works and even are competitive to some relevant Lidar-based methods.

show abstract

Hybrid conditional random field based camera-LIDAR fusion for road detection

Cited by 145 publications

References 17 publications

ChipNet: Real-Time LiDAR Processing for Drivable Region Segmentation on an FPGA

ChipNet: Real-Time LiDAR Processing for Drivable Region Segmentation on an FPGA

Road segmentation with image-LiDAR data fusion in deep neural network

Gaussian process regression-based robust free space detection for autonomous vehicle by 3-D point cloud and 2-D appearance information fusion

Contact Info

Product

Resources

About