Road scene segmentation via fusing camera and lidar data

Abstract

This paper presents an approach for pixel-wise object segmentation for road scenes based on the integration of a color image and an aligned 3D point cloud. In light of the advantage of range information in object discovery, we first produce initial object hypotheses by clustering the sparse 3D point cloud. The image pixels registered to the clustered 3D points are taken as samples to learn each object's prior knowledge. The priors are represented by Gaussian Mixture Models (GMMs) of color and 3D location information only, requiring no high-level features. We further formulate the segmentation problem within a Conditional Random Field (CRF) framework, which incorporates the learned prior models, together with hard constraints placed on the registered pixels and pairwise spatial constraints to achieve final results. Our algorithm is validated on the challenging KITTI dataset which contains diverse complicated road scenarios. Both qualitative and quantitative evaluation results show the superiority of our algorithm.

Voxel-Based Neighborhood for Spatial Shape Pattern Classification of Lidar Point Clouds with Supervised Learning

“…GMM has also been applied with color and spatial features for pixel-wise segmentation of road images [21] and object and background classification in point clouds [22]. …”

“…Moreover, different machine learning descriptors have been considered (e.g., histograms [4,8] and conditional random fields [14,15]). In particular, many solutions rely on supervised learning classifiers such as Support Vector Machines (SVM) [12,16,17,18], Gaussian Processes (GP) [19,20], or Gaussian Mixture Models (GMM) [11,21,22,23]. …”

Fusion‐based holistic road scene understanding

“…Road scene understanding plays an important role in various computer vision applications, ranging from autonomous driving to urban modeling. It commonly involves multiple tasks, such as drivable road surface detection [1,2], pedestrian and vehicle detection [3,4,5,6], semantic region labeling [7,8,9,10,11,12], geometric context reasoning [13,14], and so on. Each individual task is notoriously difficult due to the complexity of natural scenarios.…”

Road detection in image by fusion laser points based on fuzzy SVM for a small ground mobile robot

“…More than one classification criteria can be used to categorize these methods. For one hand, according to sensors being used, road detection method can be categorized into camera-based [1, 4, 6, 7, 12, 14, 15, 21-23, 25, 27, 31, 33, 37-39, 43, 46], Lidar-based [2,9,20,28,35,47,45], and fusion-based [11,19,26,32,[40][41][42]. From the beginning, the camera was used for road detection [1,12,21,33], and has been extended to nowadays [6, 7, 14, 15, 22, 23, 25, 27, 31, 36-39, 43, 46].…”

“…But range sensor usually takes low resolution and small visual field compare to camera. References [11,19,26,32,[40][41][42] fuse different kind of sensors to improve adaptability and robustness of the detection algorithms. Depth camera like Kinect is also used for indoor robot to detect object [40,42] in recent years, but this kind of sensor is still not suitable for outdoor working.…”