Automated vehicle’s behavior decision making using deep reinforcement learning and high-fidelity simulation environment

Ye, Yingjun; Zhang, Xiaohui; Sun, Jian

doi:10.1016/j.trc.2019.08.011

Cited by 134 publications

(53 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among the 8 factors of MDSI, dissociative driving factor has 8 items (11,15,27,30,34,35,36,39), anxious driving factor has 7 items (4,7,10,25,31,33,40), risky driving factor has 5 items (6,20,22,24,44), angry driving factor has 5 items (3,12,19,28,43), high-velocity driving factor has 6 items (2,5,9,16,17,32), distress-reduction driving factor has 4 items (1,8,26,37), patient driving factor has 4 items (13,18,23,38), careful driving factor has 5 items (14,21,29,41,42), 4 of them are reversed items.…”

Section: B Measurement Methods and Contents Of Mdsimentioning

confidence: 99%

“…Deep reinforcement learning enables agents to make autonomous decisions in complex environments [16]. It uses deep learning and reinforcement learning to deal with high-dimensional state space, and discrete or continuous action spaces in decision-making problems [17], [13]. Agents complete reasoning, judgment, and decision-making through a Markov decision-making process and learn how to achieve decision-making control in complex scenarios after continuous interaction with the environment [18]- [22].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

DDPG-Based Decision-Making Strategy of Adaptive Cruising for Heavy Vehicles Considering Stability

et al. 2020

View full text Add to dashboard Cite

The decision-making system of intelligent vehicles is the core component of an advanced driving system for both passenger vehicles and commercial vehicles. Finding ways to improve decisionmaking strategies to suit the complex and unfamiliar environments is a standing problem for traditional rule-based methods. This paper proposes a semi-rule-based decision-making strategy for heavy intelligent vehicles based on the Deep Deterministic Policy Gradient algorithm. Firstly, according to the car-following characteristics, the problems of high dimensions and a large amount of data in vehicle action space and state space are solved by dimension reduction and interval reduction to accelerate the training process. Subsequently, an accurate three-axle vehicle load model is established to calculate the load transfer rate value and carry out active control to increase the roll stability of heavy vehicles at high-speed corners. Furthermore, the Deep Deterministic Policy Gradient algorithm is developed based on the reward function and update function to achieve adaptive cruise control objectives for heavy vehicles on different curvature roads. Finally, the effectiveness and robustness of the algorithm are verified through simulation experiments. INDEX TERMS Adaptive cruise control, autonomous driving control system, decision-making algorithm, deep reinforcement learning, heavy vehicle, vehicle stability.

show abstract

Section: B Measurement Methods and Contents Of Mdsimentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

DDPG-Based Decision-Making Strategy of Adaptive Cruising for Heavy Vehicles Considering Stability

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Another popular microscopic simulator that has been used commercially and for research also is VISSIM [43]. In [44] it is used for developing car-following behavior and lane changing decisions.…”

Section: B Simulatorsmentioning

confidence: 99%

“…The action is the acceleration command. Reward systems use the collision of the two vehicles as a failure naturally, while the performance of the agent is based on the jerk, TTC (time to collision) [63], or passenger comfort [44]. Another approach is shown in [64], where the performance of the car following agent is evaluated against real-world measurement to achieve human-like behavior.…”

Section: A Car Followingmentioning

confidence: 99%

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Aradi

2022

IEEE Trans. Intell. Transport. Syst.

345

View full text Add to dashboard Cite

Academic research in the field of autonomous vehicles has reached high popularity in recent years related to several topics as sensor technologies, V2X communications, safety, security, decision making, control, and even legal and standardization rules. Besides classic control design approaches, Artificial Intelligence and Machine Learning methods are present in almost all of these fields. Another part of research focuses on different layers of Motion Planning, such as strategic decisions, trajectory planning, and control. A wide range of techniques in Machine Learning itself have been developed, and this article describes one of these fields, Deep Reinforcement Learning (DRL). The paper provides insight into the hierarchical motion planning problem and describes the basics of DRL. The main elements of designing such a system are the modeling of the environment, the modeling abstractions, the description of the state and the perception models, the appropriate rewarding, and the realization of the underlying neural network. The paper describes vehicle models, simulation possibilities and computational requirements. Strategic decisions on different layers and the observation models, e.g., continuous and discrete state representations, grid-based, and camera-based solutions are presented. The paper surveys the state-of-art solutions systematized by the different tasks and levels of autonomous driving, such as carfollowing, lane-keeping, trajectory following, merging, or driving in dense traffic. Finally, open questions and future challenges are discussed.

show abstract

“…Another group of researches focus on strategic decisions, where the agent determines high-level actions, such as lane-change, follow, etc. These researches usually use microscopic simulations for the environment, such as Vissim [23], Udacity, [24], SUMO [25], or several self-made models [26]. Though hybrid solutions exist, where the strategic and direct control meets [27], only a few papers deal with defining a path by some geometric approach an RL and then drive through it with a controller [28], [29].…”

Section: A Related Workmentioning

confidence: 99%

Hierarchical Evasive Path Planning Using Reinforcement Learning and Model Predictive Control

2020

View full text Add to dashboard Cite

Motion planning plays an essential role in designing self-driving functions for connected and autonomous vehicles. The methods need to provide a feasible trajectory for the vehicle to follow, fulfilling different requirements, such as safety, efficiency, and passenger comfort. In this area, algorithms must also meet strict real-time expectations, since, especially in an emergency, the decision time is limited, which raises a trade-off for the feasibility requirements. This article proposes a hierarchical path planning solution for evasive maneuvering, where a Twin Delayed DDPG reinforcement learning agent generates the parameters of a geometric path consisting of chlotoids and straight sections, and an underlying model predictive control loop fulfills the trajectory following tasks. The method is applied to the automotive double lane-change test, a common emergency situation, comparing its results with human drivers' performance using a dynamic simulation environment. Besides the test's standardized parameters, a broader range of topological layouts is chosen, both for the training and performance evaluation. The results show that the proposed method highly outperforms human drivers, especially in challenging situations, while meeting the computational requirements, as the pre-trained neural network and path generation algorithm can provide a solution in an instant, based on the experience gained during the training process.

show abstract

Automated vehicle’s behavior decision making using deep reinforcement learning and high-fidelity simulation environment

Cited by 134 publications

References 35 publications

DDPG-Based Decision-Making Strategy of Adaptive Cruising for Heavy Vehicles Considering Stability

DDPG-Based Decision-Making Strategy of Adaptive Cruising for Heavy Vehicles Considering Stability

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Hierarchical Evasive Path Planning Using Reinforcement Learning and Model Predictive Control

Contact Info

Product

Resources

About