Autonomous Driving using Safe Reinforcement Learning by Incorporating a Regret-based Human Lane-Changing Decision Model

Chen, Dong; Jiang, Longsheng; Wang, Yue; Li, Zhaojian

doi:10.23919/acc45564.2020.9147626

Cited by 51 publications

(26 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, most of the decision making and control systems are expected to meet multiple independent requirements. For instance, incorporation of a human decision making model in RL for safe and efficient plant control was described in [35], which is built on data gathered from human performance. In [36], the RL learned comfortable and safe control in multiagent scenario.…”

Section: Related Workmentioning

confidence: 99%

A Safety-Critical Decision Making and Control Framework Combining Machine Learning and Rule-based Algorithms

Aksjonov¹,

Kyrki²

2022

Preprint

View full text Add to dashboard Cite

While artificial-intelligence-based methods suffer from lack of transparency, rule-based methods dominate in safety-critical systems. Yet, the latter cannot compete with the first ones in robustness to multiple requirements, for instance, simultaneously addressing safety, comfort, and efficiency. Hence, to benefit from both methods they must be joined in a single system. This paper proposes a decision making and control framework, which profits from advantages of both the rule-and machine-learning-based techniques while compensating for their disadvantages. The proposed method embodies two controllers operating in parallel, called Safety and Learned. A rule-based switching logic selects one of the actions transmitted from both controllers. The Safety controller is prioritized every time, when the Learned one does not meet the safety constraint, and also directly participates in the safe Learned controller training. Decision making and control in autonomous driving is chosen as the system case study, where an autonomous vehicle learns a multi-task policy to safely cross an unprotected intersection. Multiple requirements (i.e., safety, efficiency, and comfort) are set for vehicle operation. A numerical simulation is performed for the proposed framework validation, where its ability to satisfy the requirements and robustness to changing environment is successfully demonstrated.

show abstract

Section: Related Workmentioning

confidence: 99%

A Safety-Critical Decision Making and Control Framework Combining Machine Learning and Rule-based Algorithms

Aksjonov¹,

Kyrki²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…As human-driven vehicles and autonomous vehicles coexist on land, efficient maneuver of autonomous vehicles has become a necessity. In [ 19 ], a regret theory is adapted based on human drivers’ lane-changing behavior. The predicted decision is integrated, and DDQN is used in training the autonomous vehicle controller.…”

Section: Related Workmentioning

confidence: 99%

Double Deep Q-Learning and Faster R-CNN-Based Autonomous Vehicle Navigation and Obstacle Avoidance in Dynamic Environment

Issa

Das

Rahman

et al. 2021

Sensors

View full text Add to dashboard Cite

Autonomous vehicle navigation in an unknown dynamic environment is crucial for both supervised- and Reinforcement Learning-based autonomous maneuvering. The cooperative fusion of these two learning approaches has the potential to be an effective mechanism to tackle indefinite environmental dynamics. Most of the state-of-the-art autonomous vehicle navigation systems are trained on a specific mapped model with familiar environmental dynamics. However, this research focuses on the cooperative fusion of supervised and Reinforcement Learning technologies for autonomous navigation of land vehicles in a dynamic and unknown environment. The Faster R-CNN, a supervised learning approach, identifies the ambient environmental obstacles for untroubled maneuver of the autonomous vehicle. Whereas, the training policies of Double Deep Q-Learning, a Reinforcement Learning approach, enable the autonomous agent to learn effective navigation decisions form the dynamic environment. The proposed model is primarily tested in a gaming environment similar to the real-world. It exhibits the overall efficiency and effectiveness in the maneuver of autonomous land vehicles.

show abstract

“…Furthermore, in the presence of multiple AVs, the AVs are expected to collaboratively learn a policy to adapt to HDVs and enable safe and efficient lane changes. As HDVs bring unknown/uncertain behaviors, planning, and control in such mixed traffic to realize safe and efficient maneuvers is a challenging task [4]. Recently, reinforcement learning (RL) has emerged as a promising framework for autonomous driving due to its online adaptation capabilities and the ability to solve complex problems [5,6].…”

Section: Introductionmentioning

confidence: 99%

“…Recently, reinforcement learning (RL) has emerged as a promising framework for autonomous driving due to its online adaptation capabilities and the ability to solve complex problems [5,6]. Several recent studies have explored the use of RL in AV lane-changing [4,7,8], which consider a single AV setting where the ego vehicle learns a lanechanging behavior by taking all other vehicles as part of the driving environment for decision making. While this single-agent approach is completely scalable, it will lead to unsatisfactory performance in the complex environment like multi-AV lane-changing in mixed traffic that requires close collaboration and coordination among AVs [9].…”

Section: Introductionmentioning

confidence: 99%

Multi-agent reinforcement learning for cooperative lane changing of connected and autonomous vehicles in mixed traffic

Zhou

Chen

Yan

et al. 2022

Auton. Intell. Syst.

Self Cite

View full text Add to dashboard Cite

Autonomous driving has attracted significant research interests in the past two decades as it offers many potential benefits, including releasing drivers from exhausting driving and mitigating traffic congestion, among others. Despite promising progress, lane-changing remains a great challenge for autonomous vehicles (AV), especially in mixed and dynamic traffic scenarios. Recently, reinforcement learning (RL) has been widely explored for lane-changing decision makings in AVs with encouraging results demonstrated. However, the majority of those studies are focused on a single-vehicle setting, and lane-changing in the context of multiple AVs coexisting with human-driven vehicles (HDVs) have received scarce attention. In this paper, we formulate the lane-changing decision-making of multiple AVs in a mixed-traffic highway environment as a multi-agent reinforcement learning (MARL) problem, where each AV makes lane-changing decisions based on the motions of both neighboring AVs and HDVs. Specifically, a multi-agent advantage actor-critic (MA2C) method is proposed with a novel local reward design and a parameter sharing scheme. In particular, a multi-objective reward function is designed to incorporate fuel efficiency, driving comfort, and the safety of autonomous driving. A comprehensive experimental study is made that our proposed MARL framework consistently outperforms several state-of-the-art benchmarks in terms of efficiency, safety, and driver comfort.

show abstract

Autonomous Driving using Safe Reinforcement Learning by Incorporating a Regret-based Human Lane-Changing Decision Model

Cited by 51 publications

References 18 publications

A Safety-Critical Decision Making and Control Framework Combining Machine Learning and Rule-based Algorithms

A Safety-Critical Decision Making and Control Framework Combining Machine Learning and Rule-based Algorithms

Double Deep Q-Learning and Faster R-CNN-Based Autonomous Vehicle Navigation and Obstacle Avoidance in Dynamic Environment

Multi-agent reinforcement learning for cooperative lane changing of connected and autonomous vehicles in mixed traffic

Contact Info

Product

Resources

About