Detecting Lane and Road Markings at A Distance with Perspective Transformer Layers

Abstract

Accurate detection of lane and road markings is a task of great importance for intelligent vehicles. In existing approaches, the detection accuracy often degrades with the increasing distance. This is due to the fact that distant lane and road markings occupy a small number of pixels in the image, and scales of lane and road markings are inconsistent at various distances and perspectives. The Inverse Perspective Mapping (IPM) can be used to eliminate the perspective distortion, but the inherent interpolation can lead to artifacts especially around distant lane and road markings and thus has a negative impact on the accuracy of lane marking detection and segmentation. To solve this problem, we adopt the Encoder-Decoder architecture in Fully Convolutional Networks and leverage the idea of Spatial Transformer Networks to introduce a novel semantic segmentation neural network. This approach decomposes the IPM process into multiple consecutive differentiable homographic transform layers, which are called "Perspective Transformer Layers". Furthermore, the interpolated feature map is refined by subsequent convolutional layers thus reducing the artifacts and improving the accuracy. The effectiveness of the proposed method in lane marking detection is validated on two public datasets: TuSimple and ApolloScape.

Development of an Autonomous Driving Vehicle for Garbage Collection in Residential Areas

“…Neven et al [ 1 ] presented lane recognition by constructing a LaneNet that can immediately obtain the next-best instance segmentation using the clustering loss function. CNN in Yu et al [ 6 ] was constructed by adding a perspective transform layer that enables accurate semantic segmentation of the lane, even when the pixels are reduced according to the distance of the lane seen in the image. Zheng et al [ 5 ] showed a recurrent feature-shift aggregator (RESA) to acquire enriched lane features from general CNN features using the spatial relationships of pixels across rows and columns.…”

“…Recently, with the rapid development of deep learning, the technology for recognizing the surrounding environment through sensor data has also developed significantly. These deep learning technologies are commonly used for recognizing lanes through cameras [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 ] or for recognizing surrounding objects [ 11 , 12 , 13 , 14 ], or as a method for simultaneously recognizing surrounding objects using the camera and lidar [ 15 , 16 ] in autonomous vehicles. Along with various high-precision sensors and deep learning technologies described above, research and commercialization of autonomous vehicles that can autonomously drive on roads without human intervention are also rapidly progressing.…”