Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving

Academic research in the field of autonomous vehicles has reached high popularity in recent years related to several topics as sensor technologies, V2X communications, safety, security, decision making, control, and even legal and standardization rules. Besides classic control design approaches, Artificial Intelligence and Machine Learning methods are present in almost all of these fields. Another part of research focuses on different layers of Motion Planning, such as strategic decisions, trajectory planning, and control. A wide range of techniques in Machine Learning itself have been developed, and this article describes one of these fields, Deep Reinforcement Learning (DRL). The paper provides insight into the hierarchical motion planning problem and describes the basics of DRL. The main elements of designing such a system are the modeling of the environment, the modeling abstractions, the description of the state and the perception models, the appropriate rewarding, and the realization of the underlying neural network. The paper describes vehicle models, simulation possibilities and computational requirements. Strategic decisions on different layers and the observation models, e.g., continuous and discrete state representations, grid-based, and camera-based solutions are presented. The paper surveys the state-of-art solutions systematized by the different tasks and levels of autonomous driving, such as carfollowing, lane-keeping, trajectory following, merging, or driving in dense traffic. Finally, open questions and future challenges are discussed.

show abstract

“…and MOBIL [40] model. In the other publication from the same author [100], the action space is changed slightly by changing [65] Fig . 12.…”

Section: Driving In Trafficmentioning

confidence: 99%

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Aradi

2022

IEEE Trans. Intell. Transport. Syst.

343

View full text Add to dashboard Cite

show abstract

“…We use a network architecture similar to Hoel et al [9]. Our early experiments showed that training was faster and more stable using this architecture compared to standard fully connected networks.…”

Section: Policy Networkmentioning

confidence: 99%

“…3. Illustration of the network architecture [9]. The convolutional layers allow to share the weights when processing other vehicles' states.…”

Section: Policy Networkmentioning

confidence: 99%

Reinforcement Learning with Iterative Reasoning for Merging in Dense Traffic

Bouton

Nakhaei²,

Isele

et al. 2020

2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC)

Self Cite

View full text Add to dashboard Cite

Maneuvering in dense traffic is a challenging task for autonomous vehicles because it requires reasoning about the stochastic behaviors of many other participants. In addition, the agent must achieve the maneuver within a limited time and distance. In this work, we propose a combination of reinforcement learning and game theory to learn merging behaviors. We design a training curriculum for a reinforcement learning agent using the concept of level-k behavior. This approach exposes the agent to a broad variety of behaviors during training, which promotes learning policies that are robust to model discrepancies. We show that our approach learns more efficient policies than traditional training methods.

show abstract

“…Since Deepmind's team used recent advances in training deep neural networks to develop a novel artificial agent [16]- [18], termed a deep Q-network [19], which was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester, deep reinforcement learning (DRL) has become a useful method to solve control problems and decision-making problems [20], [21]. Unlike the above decision-making method, DRL decision-making does not need to make trial maneuvers and predict the positions at each decision-making stage.…”

Section: Introductionmentioning

confidence: 99%

Maneuver Decision-Making of Deep Learning for UCAV Thorough Azimuth Angles

Zhang

Huang

2020

IEEE Access

View full text Add to dashboard Cite

Maneuver decision-making directly determines the success or failure of air combat. To improve the dogfight ability of unmanned combat aerial vehicles and avoid the deficiencies of traditional methods, such as poor flexibility and a weak decision-making ability, a maneuver method using deep learning is proposed. A total of 72 different maneuvers are constructed, and 544320 states are designed. Flight simulations are conducted under these different states to obtain corresponding future azimuth angles. A deep neural network is trained with these offline data, and thus, the network possesses state prediction capability. A situation assessment function and a decision objective function based on azimuth angles are constructed. During air combat, the optimal maneuver is selected from the maneuver library according to the predicted state and the decision objective function. The results of air combat simulations indicate that the unmanned combat aerial vehicle (UCAV) can win the air combat game by the proposed method in a balanced situation and can meet missile launching conditions in an adverse situation. The operational time of this method has been reduced by 0.01 s compared with the comparison method.

show abstract

Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving

Cited by 222 publications

References 27 publications

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Reinforcement Learning with Iterative Reasoning for Merging in Dense Traffic

Maneuver Decision-Making of Deep Learning for UCAV Thorough Azimuth Angles

Contact Info

Product

Resources

About