Automated Lane Change Decision Making using Deep Reinforcement Learning in Dynamic and Uncertain Highway Environment

Alizadeh, Ali; Moghadam, Majid; Bicer, Yunus; Üre, Nazım Kemal; Yavaş, Uğur; Kurtulus, Can

doi:10.48550/arxiv.1909.11538

Cited by 3 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In lateral direction, we penalize the lateral distance to the target lane center. In longitudinal direction, we predict constant velocity trajectories for all relevant 1 surrounding vehicles, so that we can maintain safe distance between them and the RL agent. This is done by calculating the time-to-collision and time-headway values for the planned trajectory, relative to the relevant surrounding vehicles.…”

Section: B Trajectory Planningmentioning

confidence: 99%

“…In the discrete actions case (e.g. [1,6,31] and also [13,7,8,9,10]), the agent can choose from actions such as keep lane, lanechange to the left or lane-change to the right or fixed accelerating/decelerating steps. While the small and fixed action set leads to fast learning progress, the lane-change maneuvers are usually with fixed execution duration, resulting in a suboptimal, unnatural behavior in tight situations.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Amortized Q-learning with Model-based Action Proposals for Autonomous Driving on Highways

Mirchevska¹,

Hügle²,

Kalweit³

et al. 2020

Preprint

View full text Add to dashboard Cite

Well-established optimization-based methods can guarantee an optimal trajectory for a short optimization horizon, typically no longer than a few seconds. As a result, choosing the optimal trajectory for this short horizon may still result in a sub-optimal long-term solution. At the same time, the resulting short-term trajectories allow for effective, comfortable and provable safe maneuvers in a dynamic traffic environment. In this work, we address the question of how to ensure an optimal long-term driving strategy, while keeping the benefits of classical trajectory planning. We introduce a Reinforcement Learning based approach that coupled with a trajectory planner, learns an optimal long-term decision-making strategy for driving on highways. By online generating locally optimal maneuvers as actions, we balance between the infinite low-level continuous action space, and the limited flexibility of a fixed number of predefined standard lane-change actions. We evaluated our method on realistic scenarios in the opensource traffic simulator SUMO and were able to achieve better performance than the 4 benchmark approaches we compared against, including a random action selecting agent, greedy agent, high-level, discrete actions agent and an IDM-based SUMO-controlled agent.

show abstract

Section: B Trajectory Planningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Amortized Q-learning with Model-based Action Proposals for Autonomous Driving on Highways

Mirchevska¹,

Hügle²,

Kalweit³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

Amortized Q-learning with Model-based Action Proposals for Autonomous Driving on Highways

Mirchevska

Hügle

Kalweit

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Development of A Stochastic Traffic Environment with Generative Time-Series Models for Improving Generalization Capabilities of Autonomous Driving Agents

Ozturk

Gunel

Dal

et al. 2020

2020 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

Automated lane changing is a critical feature for advanced autonomous driving systems. In recent years, reinforcement learning (RL) algorithms trained on traffic simulators yielded successful results in computing lane changing policies that strike a balance between safety, agility and compensating for traffic uncertainty. However, many RL algorithms exhibit simulator bias and policies trained on simple simulators do not generalize well to realistic traffic scenarios. In this work, we develop a data driven traffic simulator by training a generative adverserial network (GAN) on real life trajectory data. The simulator generates randomized trajectories that resembles real life traffic interactions between vehicles, which enables training the RL agent on much richer and realistic scenarios. We demonstrate through simulations that RL agents that are trained on GAN-based traffic simulator has stronger generalization capabilities compared to RL agents trained on simple rule-driven simulators.

show abstract

Automated Lane Change Decision Making using Deep Reinforcement Learning in Dynamic and Uncertain Highway Environment

Cited by 3 publications

References 13 publications

Amortized Q-learning with Model-based Action Proposals for Autonomous Driving on Highways

Amortized Q-learning with Model-based Action Proposals for Autonomous Driving on Highways

Amortized Q-learning with Model-based Action Proposals for Autonomous Driving on Highways

Development of A Stochastic Traffic Environment with Generative Time-Series Models for Improving Generalization Capabilities of Autonomous Driving Agents

Contact Info

Product

Resources

About