An End-to-end Deep Reinforcement Learning Approach for the Long-term Short-term Planning on the Frenet Space

Abstract

Tactical decision making and strategic motion planning for autonomous highway driving are challenging due to the complication of predicting other road users' behaviors, diversity of environments, and complexity of the traffic interactions. This paper presents a novel end-to-end continuous deep reinforcement learning approach towards autonomous cars' decision-making and motion planning. For the first time, we define both states and action spaces on the Frenet space to make the driving behavior less variant to the road curvatures than the surrounding actors' dynamics and traffic interactions. The agent receives time-series data of past trajectories of the surrounding vehicles and applies convolutional neural networks along the time channels to extract features in the backbone. The algorithm generates continuous spatiotemporal trajectories on the Frenet frame for the feedback controller to track. Extensive high-fidelity highway simulations on CARLA show the superiority of the presented approach compared with commonly used baselines and discrete reinforcement learning on various traffic scenarios. Furthermore, the proposed method's advantage is confirmed with a more comprehensive performance evaluation against 1000 randomly generated test scenarios.

An automatic driving trajectory planning approach in complex traffic scenarios based on integrated driver style inference and deep reinforcement learning

“…Directly issuing control commands using reinforcement learning algorithms can introduce safety risks due to the black-box nature of the approach. Consequently, in recent years, scholars have proposed end-to-end motion planning, employing reinforcement learning to generate driving trajectories as actions while ensuring safety, thus maximizing the advantages of end-to-end methods [26].…”

Intelligent Vehicle Decision-Making and Trajectory Planning Method Based on Deep Reinforcement Learning in the Frenet Space

“…In the overarching framework of intelligent driving [ 27 ], the task of trajectory planning resides downstream of behavior decision making and shoulders the responsibility of translating extended decision-making objectives into specific vehicle driving paths within predefined temporal windows [ 28 ]. These driving trajectories encapsulate the vehicle position and velocity data at discrete time intervals [ 29 ].…”

Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA Simulator

“…With the last year’s improvements in deep learning algorithms and the modern hardware computation capabilities, researchers at NVIDIA proposed an end-to-end supervised learning method based on CNN capable of directly steering an automobile [ 19 ]. From this pioneering work, many others have used this approach [ 20 , 21 ].…”