Overtaking Maneuvers in Simulated Highway Driving using Deep Reinforcement Learning

Kaushik, Meha; Prasad, V.; Krishna, K. Madhava; Ravindran, Balaraman

doi:10.1109/ivs.2018.8500718

Cited by 73 publications

(42 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Curriculum learning describes a type of learning in which the training starts with only easy examples of a task and then gradually increase difficulty. This approach is used in [18]- [20]. • Adversarial learning aims to fool models through malicious input.…”

Section: B Reinforcement Learningmentioning

confidence: 99%

“…The surrounding from the perspective of the vehicle can be described by a coarse perception map where the target is represented by a red dot (c) (source: [78]). since there is no reflection, which is provided by TORCS and used in [20], is to represent the lane markings with imagined beam sensors. The agent in the cited example uses readings from 19 sensors with a 200m range, presenting at every 10 • on the front half of the car returning distance to the track edge.…”

Section: E Observation Spacementioning

confidence: 99%

See 1 more Smart Citation

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Aradi

2022

IEEE Trans. Intell. Transport. Syst.

348

View full text Add to dashboard Cite

Academic research in the field of autonomous vehicles has reached high popularity in recent years related to several topics as sensor technologies, V2X communications, safety, security, decision making, control, and even legal and standardization rules. Besides classic control design approaches, Artificial Intelligence and Machine Learning methods are present in almost all of these fields. Another part of research focuses on different layers of Motion Planning, such as strategic decisions, trajectory planning, and control. A wide range of techniques in Machine Learning itself have been developed, and this article describes one of these fields, Deep Reinforcement Learning (DRL). The paper provides insight into the hierarchical motion planning problem and describes the basics of DRL. The main elements of designing such a system are the modeling of the environment, the modeling abstractions, the description of the state and the perception models, the appropriate rewarding, and the realization of the underlying neural network. The paper describes vehicle models, simulation possibilities and computational requirements. Strategic decisions on different layers and the observation models, e.g., continuous and discrete state representations, grid-based, and camera-based solutions are presented. The paper surveys the state-of-art solutions systematized by the different tasks and levels of autonomous driving, such as carfollowing, lane-keeping, trajectory following, merging, or driving in dense traffic. Finally, open questions and future challenges are discussed.

show abstract

Section: B Reinforcement Learningmentioning

confidence: 99%

Section: E Observation Spacementioning

confidence: 99%

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Aradi

2022

IEEE Trans. Intell. Transport. Syst.

348

View full text Add to dashboard Cite

show abstract

“…The RL algorithm was based on DDPG and applied to the navigation task. Kaushik, et al [14] also used DDPG to learn overtaking maneuvers in continuous action space in a way of curriculum learning that made the agent first learn simple tasks (lane keeping) and then moved on to complex tasks (overtaking), with the goal of fast overtaking car in front of the RL vehicle. Sallab, et al [15] compared the effect of using discrete action space and continuous action space for the lane keeping task, with the use of deep Qnetwork and Deep Deterministic Actor Critic (DDAC) respectively, and concluded that both methods could achieve successful lane keeping behavior but DDAC showed better performance with smoothed actions.…”

Section: Related Workmentioning

confidence: 99%

Continuous Control for Automated Lane Change Behavior Based on Deep Deterministic Policy Gradient Algorithm

Wang

Chan

2019

2019 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

Lane change is a challenging task which requires delicate actions to ensure safety and comfort. Some recent studies have attempted to solve the lane-change control problem with Reinforcement Learning (RL), yet the action is confined to discrete action space. To overcome this limitation, we formulate the lane change behavior with continuous action in a model-free dynamic driving environment based on Deep Deterministic Policy Gradient (DDPG). The reward function, which is critical for learning the optimal policy, is defined by control values, position deviation status, and maneuvering time to provide the RL agent informative signals. The RL agent is trained from scratch without resorting to any prior knowledge of the environment and vehicle dynamics since they are not easy to obtain. Seven models under different hyperparameter settings are compared. A video showing the learning progress of the driving behavior is available 2 . It demonstrates the RL vehicle agent initially runs out of road boundary frequently, but eventually has managed to smoothly and stably change to the target lane with a success rate of 100% under diverse driving situations in simulation.

show abstract

“…Prior works on reinforcement learning for autonomous driving that used fully-connected network architectures and fixed sized inputs [6], [7], [5], [8], [9] are limited in the number of vehicles that can be considered. CNNs using occupancy grids [10], [11] are limited to their initial grid size.…”

Section: Introductionmentioning

confidence: 99%

Dynamic Input for Deep Reinforcement Learning in Autonomous Driving

Huegle

Kalweit

Mirchevska

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

The common pipeline in autonomous driving systems is highly modular and includes a perception component which extracts lists of surrounding objects and passes these lists to a high-level decision component. In this case, leveraging the benefits of deep reinforcement learning for high-level decision making requires special architectures to deal with multiple variable-length sequences of different object types, such as vehicles, lanes or traffic signs. At the same time, the architecture has to be able to cover interactions between traffic participants in order to find the optimal action to be taken. In this work, we propose the novel Deep Scenes architecture, that can learn complex interaction-aware scene representations based on extensions of either 1) Deep Sets or 2) Graph Convolutional Networks. We present the Graph-Q and DeepScene-Q off-policy reinforcement learning algorithms, both outperforming state-ofthe-art methods in evaluations with the publicly available traffic simulator SUMO.

show abstract

Overtaking Maneuvers in Simulated Highway Driving using Deep Reinforcement Learning

Cited by 73 publications

References 8 publications

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Continuous Control for Automated Lane Change Behavior Based on Deep Deterministic Policy Gradient Algorithm

Dynamic Input for Deep Reinforcement Learning in Autonomous Driving

Contact Info

Product

Resources

About