Real-Time Road Segmentation Using LiDAR Data Processing on an FPGA

Abstract

This paper presents the FPGA design of a convolutional neural network (CNN) based road segmentation algorithm for real-time processing of LiDAR data. For autonomous vehicles, it is important to perform road segmentation and obstacle detection such that the drivable region can be identified for path planning. Traditional road segmentation algorithms are mainly based on image data from cameras, which is subjected to the light condition as well as the quality of road markings. LiDAR sensor can obtain the 3D geometry information of the vehicle surroundings with very high accuracy. However, it is a computational challenge to process a large amount of LiDAR data at real-time. In this work, a convolutional neural network model is proposed and trained to perform semantic segmentation using the LiDAR sensor data. Furthermore, an efficient hardware design is implemented on the FPGA that can process each LiDAR scan in 16.9ms, which is much faster than the previous works. Evaluated using KITTI road benchmarks, the proposed solution achieves high accuracy of road segmentation.

FPGA-Based Inter-layer Pipelined Accelerators for Filter-Wise Weight-Balanced Sparse Fully Convolutional Networks with Overlapped Tiling

“…These networks are not suitable for embedded systems because of their network size and complexity. Thus, some studies proposed specific lightweight networks [11,32] for semantic segmentation to be implemented on an FPGA. Lyu et al [32] proposed a small network for road segmentation with light detection and ranging (LiDAR), and its FPGA implementation meets a real-time processing requirement (59.2 FPS).…”

“…Thus, some studies proposed specific lightweight networks [11,32] for semantic segmentation to be implemented on an FPGA. Lyu et al [32] proposed a small network for road segmentation with light detection and ranging (LiDAR), and its FPGA implementation meets a real-time processing requirement (59.2 FPS). In exchange for the FPGA realization by the proposed small model, it can deal with road class only, and therefore the implementation challenges of the task for many categories still remain.…”

LodoNet

“…In this work, however, the LiDAR points are not uniformly distributed on the ground plane but heavily gathered together near the LiDAR scanner, which results in massive dropped points in the near-range and redundant space in the far-range. Lyu et al [17,18] and RangeNet++ [19] improve the projection scheme by replacing the target plane with a sphere surface, in which LiDAR points are nearly uniformly distributed. SqueezeSeg V1 [33], V2 [34], and LO-Net [12] also employ this projection scheme and result in a good performance in LiDAR point semantic segmentation.…”

“…Fortunately, Lyu et al [17,18] and RangeNet++ [19] have introduced an algorithm to project the LiDAR data on to a spherical view so that a LiDAR point cloud with geometry features can be transferred to an image-like feature map with minor point losses. By employing this projection, we can efficiently generate image representations of LiDAR frames for feature extraction.…”

An Interactive LiDAR to Camera Calibration

“…Automated driving systems (ADS) and advanced driver assistant systems (ADAS) equipped on intelligent vehicles rely on multiple sensors to perceive their surroundings. In recent research works, LiDAR-based algorithms have shown their advantage on drivable region segmentation [8] [9], object detection [18], and simultaneous localization and mapping [19] [15]. LIDARs are also fused with cameras to improve the accuracy of 3D object detection [2].…”