End-to-End Deep Learning of Lane Detection and Path Prediction for Real-Time Autonomous Driving

Lee, Der-Hau; Liu, Jinn-Liang

doi:10.48550/arxiv.2102.04738

Cited by 2 publications

(10 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is also an important step towards safety measures and interpretability of end-to-end driving systems [1,2,3,4]. We propose such an integrated algorithm consisting of a new UNet architecture for multi-task learning, a path prediction model [23] using UNet's output, a modified lateral controller, and a modeland learning-based longitudinal controller.…”

Section: Contributionmentioning

confidence: 99%

“…We reduce 5 indicators in our previous work [5] to 2 (θ and ∆ [21,23]), change one overtaking classification in [5] to two pose classifications, and add a lane semantic segmentation [23] in the present work. The driving tasks are different between these two works.…”

Section: Contributionmentioning

confidence: 99%

“…Figure 1 illustrates the architecture of our MTDNN using UNets, ResUNet [49,51] for example, as a backbone that can perform multiple (segmentation, regression, and classification) tasks for controllers to make steering, acceleration, and braking decisions. We refer to [23] for a brief survey of UNets related to autonomous driving and other applications.…”

Section: Multi-task Unetsmentioning

confidence: 99%

“…The output of the segmentation subnet has the same resolution of the input image and a single channel from a pixel-wise sigmoid operation that yields a gray (probability) image (GImg) of lane lines [23] for the segmentation task. The normalized cross entropy loss [68]…”

Section: Multi-task Unetsmentioning

confidence: 99%

“…The system is a pipeline consisting of perception sensors, DNN, and control actuators [1,2,3] with a data flow from sensors to DNN to path planning to controllers to actuators for making driving decisions of steering, acceleration, or braking in an end-to-end, autonomous, and real-time manner [1,2,3,5]. Since Pomerleau's pioneering work in the 1980s [6], a variety of end-to-end DNNs have been proposed for various tasks in autonomous driving [5,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24]. Most of these DNNs belong to singletask learning models having single (regression or probabilistic) loss function for training the model to infer single driving task (steering angle, lead car's distance, or turning etc.)…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

End-to-End Multi-Task Deep Learning and Model Based Control Algorithm for Autonomous Driving

Lee¹,

Liu²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

End-to-end driving with a deep learning neural network (DNN) has become a rapidly growing paradigm of autonomous driving in industry and academia. Yet safety measures and interpretability still pose challenges to this paradigm. We propose an end-toend driving algorithm that integrates multi-task DNN, path prediction, and control models in a pipeline of data flow from sensory devices through these models to driving decisions. It provides quantitative measures to evaluate the holistic, dynamic, and real-time performance of end-to-end driving systems, and thus allows to quantify their safety and interpretability. The DNN is a modified UNet, a well known encoder-decoder neural network of semantic segmentation. It consists of one segmentation, one regression, and two classification tasks for lane segmentation, path prediction, and vehicle controls. We present three variants of the modified UNet architecture having different complexities, compare them on different tasks in four static measures for both single and multi-task (MT) architectures, and then identify the best one by two additional dynamic measures in real-time simulation. We also propose a learning-and model-based longitudinal controller using model predictive control method. With the Stanley lateral controller, our results show that MTUNet outperforms an earlier modified UNet in terms of curvature and lateral offset estimation on curvy roads at normal speed, which has been tested in a real car driving on real roads.

show abstract

Section: Contributionmentioning

confidence: 99%

Section: Contributionmentioning

confidence: 99%

Section: Multi-task Unetsmentioning

confidence: 99%

Section: Multi-task Unetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

End-to-End Multi-Task Deep Learning and Model Based Control Algorithm for Autonomous Driving

Lee¹,

Liu²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Deep-Learning-Based Network for Lane Following in Autonomous Vehicles

2022

View full text Add to dashboard Cite

The research field of autonomous self-driving vehicles has recently become increasingly popular. In addition, motion-planning technology is essential for autonomous vehicles because it mitigates the prevailing on-road obstacles. Herein, a deep-learning-network-based architecture that was integrated with VGG16 and the gated recurrent unit (GRU) was applied for lane-following on roads . The normalized input image was fed to the three-layer VGG16 output layer as a pattern and the GRU output layer as the last layer. Next, the processed data were fed to the two fully connected layers, with a dropout layer added in between each layer. Afterward, to evaluate the deep-learning-network-based model, the steering angle and speed from the control task were predicted as output parameters. Experiments were conducted using the a dataset from the Udacity simulator and a real dataset. The results show that the proposed framework remarkably predicted steering angles in different directions. Furthermore, the proposed approach achieved higher mean square errors of 0.0230 and 0.0936 and and inference times of 3–4 and 3 ms. We also implemented our proposed framework on the NVIDIA Jetson embedded platform (Jetson Nano 4 GB) and compared it with the GPU’s computational time. The results revealed that the embedded system took 45–46 s to execute a single epoch in order to predict the steering angle . The results show that the proposed framework generates fruitful and accurate motion planning for lane-following in autonomous driving.

show abstract

End-to-End Deep Learning of Lane Detection and Path Prediction for Real-Time Autonomous Driving

Cited by 2 publications

References 9 publications

End-to-End Multi-Task Deep Learning and Model Based Control Algorithm for Autonomous Driving

End-to-End Multi-Task Deep Learning and Model Based Control Algorithm for Autonomous Driving

Deep-Learning-Based Network for Lane Following in Autonomous Vehicles

Contact Info

Product

Resources

About