Continuous decision‐making for autonomous driving at intersections using deep deterministic policy gradient

Abstract

Intersections have been identified as the most complex and accident-prone traffic scenarios on road. Making appropriate decisions at intersections for driving safety, efficiency, and comfort become a challenging task for autonomous vehicles (AVs). The existing research on AV decision-making at intersections either considers a single scenario only with discrete behaviour outputs or ignores the requirements for driving efficiency and comfort. To address these problems, this study proposed a deep reinforcement learning based continuous decision-making method to make AVs drive through intersections. The proposed method establishes an end-to-end decision-making framework by using a convolutional neural network to map the relationship between traffic images and vehicle operations. The interaction between the AV and other vehicles was modelled as a Markov decision process (MDP), and a deep deterministic policy gradient algorithm was employed to solve the MDP problem and obtain the optimal driving policy. The top three accident-prone crossing path crash scenarios at intersections were realized in CARLA to verify the effectiveness of the proposed method. The experimental results demonstrated that the developed method could provide effective policies to ensure driving safety and efficiency while considering driving comfort for autonomous driving at intersections in all the examined scenarios.This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

D-HAL: Distributed Hierarchical Adversarial Learning for Multi-Agent Interaction in Autonomous Intersection Management

“…In contrast, the learning-based method is more flexible in execution. The dominant method in learning-based AIM is multi-agent reinforcement learning (MARL), including Deep Q Learning (DQN) [39][40][41][42][43], Dueling Deep Q Learning (DDQN) [44], Deep Deterministic Policy Gradient (DDPG) [45], Proximal Policy Optimization (PPO) [46][47][48], Twin Delayed Deep Deterministic Policy Gradient (TD3) [49][50][51], and Soft Actor-Critic (SAC) [52]. Besides, DCL-AIM introduced coordinate state and independent state for CAVs to react in different scenarios [53], RAIM [54] and adv.RAIM [55] applied encoder-decoder structure with LSTM cell, AIM5LA further considered communication delay based on the adv.RAIM [56], and game theory was utilized to determine the leader-follower to enhance the performance of reinforcement learning in [52,57].…”

Vision-Based Autonomous Vehicle Systems Based on Deep Learning: A Systematic Literature Review

“…Similarly, continuous decision-making for intersection cases in top three accidentprone crossing paths in a Carla simulator using DDPG and CNN surpassed the limitation of single scenario with discrete behavior outputs fulfilling the criteria for safe AVS [140]. DDQG was utilized to address the MDP problem and find the best driving strategy by mapping the link between traffic photos and vehicle operations through CNN that solved the common drawback of rule-based RL methods deployed in intersection cases.…”

Driving Conflict Resolution of Autonomous Vehicles at Unsignalized Intersections: A Differential Game Approach

“…Besides, learning-based approaches have been widely used for autonomous decision making. In [21], an end-to-end decision-making framework is designed by using a convolutional neural network to map the relationship between traffic images and vehicle operations. In [22], a safe reinforcement learning (RL) approach is proposed to address the driving conflicts at autonomously navigate intersections, which shows good robustness against perception errors and occlusions.…”