Emergency Vehicle Aware Lane Change Decision Model for Autonomous Vehicles Using Deep Reinforcement Learning

Abstract

Autonomous Vehicles (AVs) have advanced rapidly in recent years as they promise to be safe and minimize the burden coming from the driving task. AVs share the road with various categories of vehicles as Emergency Vehicles (EMVs) (e.g police and ambulance vehicles). When being approached by an active EMV, it is natural to expect all vehicles to cooperate with EMV, such that the EMV travel time is minimized. The decision-making block of an AV includes the responsibility of instructing the AV to change lanes, which is typically handled by the Lane Change Decision (LCD) model. A typical LCD model tends to overlook the presence of EMVs around, as they neglect the impact of the lane change on the EMV utility. To address this challenge , this paper proposes an Emergency Vehicle Aware LCD via utilizing Deep Reinforcement Learning . To our best knowledge, this is one of the pioneering works that propose a DRL solution for the problem, addressing important limitations that have been identified. The proposed solution was evaluated against a rule-based LCD known as MOBIL in terms of safety and level of cooperativeness with the EMV. Some key results found from the comparison between the proposed solution and MOBIL are (1) identical safety levels ,(2) proposed solution is takes far less time to give up the lane when being approached by an EMV, and (3) proposed solution never blocks the path of the EMV, whereas MOBIL occasionally block the path.

An Empirical Study of DDPG and PPO-Based Reinforcement Learning Algorithms for Autonomous Driving

“…Both the algorithms showed successful results, however, DDAC delivered better performance due to the presence of continuous policy which helped choose smoother actions. Alzubaidi et al [27] proposed an Emergency Vehicle (EMVs) aware Lane Changing Decision (LCD) model using DQN. This algorithm was used for training and testing since the addressed issue had infinite state spaces.…”

“…Li et al [10] DRL algos. proposed a framework for autonomous vehicles to make decisions in lane changing scenarios using DRL on CARLA simulator Elallid et al [25] DQN Proposed a reinforcement learning based model using DQN to control the autonomous vehicles in complex scenarios involving pedestrians and moving vehicles using CARLA simulator Sallab et al [26] DQN, DDAC combined with RNN Proposed a framework that incorporated Recurrent Neural Networks along with DRL algorithms( DQN and DDAC) enabling the car to handle partially observable scenarios Alzubaidi et al [27] DQN Proposed an EMVs aware LCD model using DQN Fayjie et al [28] DQN Proposed a DRL based approach to facilitate navigation and avoid obstacles in an urban environment. Ye et al [29] DRL algos.…”

A Collaborative Control Scheme for Smart Vehicles Based on Multi-Agent Deep Reinforcement Learning

“…Traditional recurrent neural networks cannot model longtime dependent information in traffic flow sequences [14]. Deep learning is the process of learning the intrinsic laws and levels of representation of sample data.…”

“…This algorithm uses the parameter policy to approximate the policy directly and updates the parameters of the characterized policy based on the gradient of the performance metrics [14]. In the Actor-Critic framework, the parametric strategy is characterized using the parameter Sg(x) and the value function is characterized using the parameter A i [41].…”