End-to-end learning for lane keeping of self-driving cars

Chen, Zhilu; Huang, Xinming

doi:10.1109/ivs.2017.7995975

Cited by 234 publications

(112 citation statements)

References 6 publications

Supporting

Mentioning

109

Contrasting

Unclassified

Order By: Relevance

“…We then study the effect of changing the value of x on the performance of our model in terms of RMSE. We train our model at separate x values where x is set to 1,2,4,6,8,10,12,14,20 and computed the RMSE value for both the training and validation data respectively at each x value. The results were plotted in Fig.…”

Section: Analysis and Resultsmentioning

confidence: 99%

“…Controlling the steering angle is a fundamental problem for autonomous vehicles [1], [2], [3]. Recent computer vision-based approaches to control the steering angle in autonomous cars mostly focus on improving the driving accuracy with the local data collected from the sensors on the same vehicle and as such, they consider each car as an isolated unit gathering and processing information locally.…”

Section: Introductionmentioning

confidence: 99%

“…Our experimental results demonstrate that our proposed CNN-LSTM-based model yields the state-ofthe-art results. Our main contributions are: (1) we propose an end-to-end vehicle-assisted steering angle control system for cooperative systems; (2) We propose using a large sequence of images as opposed to using only two consecutive frames;…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networks

Valiente¹,

Zaman²,

Özer

et al. 2019

2019 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

A fundamental challenge in autonomous vehicles is adjusting the steering angle at different road conditions. Recent state-of-the-art solutions addressing this challenge include deep learning techniques as they provide end-to-end solution to predict steering angles directly from the raw input images with higher accuracy. Most of these works ignore the temporal dependencies between the image frames. In this paper, we tackle the problem of utilizing multiple sets of images shared between two autonomous vehicles to improve the accuracy of controlling the steering angle by considering the temporal dependencies between the image frames. This problem has not been studied in the literature widely. We present and study a new deep architecture to predict the steering angle automatically by using Long-Short-Term-Memory (LSTM) in our deep architecture. Our deep architecture is an end-to-end network that utilizes CNN, LSTM and fully connected (FC) layers and it uses both present and futures images (shared by a vehicle ahead via Vehicle-to-Vehicle (V2V) communication) as input to control the steering angle. Our model demonstrates the lowest error when compared to the other existing approaches in the literature.

show abstract

Section: Analysis and Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networks

Valiente¹,

Zaman²,

Özer

et al. 2019

2019 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

show abstract

“…Unlike traditional approaches [25] that divide the system into two separate perception and control parts which contain tasks like lane detection, path planning and control logic, End-to-End approaches often directly learn the mapping from raw pixels to vehicle actuation. Recent demonstrations have shown some successful examples of training systems End-to-End to perform simple tasks like lane-keeping [6] or obstacle avoidance [17].…”

Section: Introductionmentioning

confidence: 99%

Learning On-Road Visual Control for Self-Driving Vehicles With Auxiliary Tasks

Chen

Palanisamy²,

Mudalige³

et al. 2019

2019 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

A safe and robust on-road navigation system is a crucial component of achieving fully automated vehicles. NVIDIA recently proposed an End-to-End algorithm that can directly learn steering commands from raw pixels of a front camera by using one convolutional neural network. In this paper, we leverage auxiliary information aside from raw images and design a novel network structure, called Auxiliary Task Network (ATN), to help boost the driving performance while maintaining the advantage of minimal training data and an End-to-End training method. In this network, we introduce human prior knowledge into vehicle navigation by transferring features from image recognition tasks. Image semantic segmentation is applied as an auxiliary task for navigation. We consider temporal information by introducing an LSTM module and optical flow to the network. Finally, we combine vehicle kinematics with a sensor fusion step. We discuss the benefits of our method over stateof-the-art visual navigation methods both in the Udacity simulation environment and on the real-world Comma.ai dataset.

show abstract

“…The traffic scene perception task can be separated into two sub-tasks: object detection and road/lane detection. Object detection includes vehicle detection [1] [2] [3] [4] [5], pedestrian detection [6] [3] [4] and traffic light/sign detection [7] [8] [9] [10] [11], while road/lane detection includes road marking detection [12] [13] [14], lane detection [15] [16] [17] and road segmentation [18] [19] [16]. In this work, we are primarily concentrating on road segmentation, since it is a fundamental component of automated driving that provides the drivable region for the vehicle's next movement.…”

Section: Introductionmentioning

confidence: 99%

Real-Time Road Segmentation Using LiDAR Data Processing on an FPGA

Lyu

Bai

Huang

2018

2018 IEEE International Symposium on Circuits and Systems (ISCAS)

Self Cite

View full text Add to dashboard Cite

This paper presents the FPGA design of a convolutional neural network (CNN) based road segmentation algorithm for real-time processing of LiDAR data. For autonomous vehicles, it is important to perform road segmentation and obstacle detection such that the drivable region can be identified for path planning. Traditional road segmentation algorithms are mainly based on image data from cameras, which is subjected to the light condition as well as the quality of road markings. LiDAR sensor can obtain the 3D geometry information of the vehicle surroundings with very high accuracy. However, it is a computational challenge to process a large amount of LiDAR data at real-time. In this work, a convolutional neural network model is proposed and trained to perform semantic segmentation using the LiDAR sensor data. Furthermore, an efficient hardware design is implemented on the FPGA that can process each LiDAR scan in 16.9ms, which is much faster than the previous works. Evaluated using KITTI road benchmarks, the proposed solution achieves high accuracy of road segmentation.

show abstract

End-to-end learning for lane keeping of self-driving cars

Cited by 234 publications

References 6 publications

Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networks

Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networks

Learning On-Road Visual Control for Self-Driving Vehicles With Auxiliary Tasks

Real-Time Road Segmentation Using LiDAR Data Processing on an FPGA

Contact Info

Product

Resources

About