A Multi-Agent Deep Reinforcement Learning Coordination Framework for Connected and Automated Vehicles at Merging Roadways

Nakka, Sai Krishna Sumanth; Chalaki, Behdad; Malikopoulos, Andreas A.

doi:10.23919/acc53348.2022.9867314

Cited by 10 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Shi et al [28] proposed a cooperative strategy of longitudinal control for a mixed connected and automated traffic environment based on the DRL algorithm and enhanced performance for an entire mixed traffic flow. Nakka et al [29] proposed a decentralized multi-agent RL (MARL) framework for coordinating CAVs in a highway merging scenario and employed an actor-critic architecture with a centralized critic and decentralized actors to avoid the problem of a non-stationary environment. Fares et al [30] applied a cooperative Q-learning algorithm based on a coordinated graph structure to optimize overall traffic congestion on a motorway through multiple ramp control.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Cooperative decision making for connected automated vehicles in multiple driving scenarios

Wang

Zhu

et al. 2022

IET Intelligent Trans Sys

View full text Add to dashboard Cite

To improve the application range of decision‐making systems for connected automated vehicles, this paper proposes a cooperative decision‐making approach for multiple driving scenarios based on the combination of multi‐agent reinforcement learning with centralized planning. Specifically, the authors derived driving tasks from driving scenarios and computed the policy functions for different driving scenarios as linear combinations of policy functions for a set of specific driving tasks. Then, the authors classified vehicle coalitions according to the relationships between vehicles and used centralized planning methods to determine the optimal combination of actions for each coalition. Finally, the authors conducted tests in two driving scenarios considering different traffic densities to evaluate the performance of the developed approach. Simulation results demonstrate that the proposed approach exhibits good robustness in multiple driving scenarios while enabling cooperative decision making for connected automated vehicles, thereby ensuring safe and rational decision making.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Nakka et al. [29] proposed a decentralized multi‐agent RL (MARL) framework for coordinating CAVs in a highway merging scenario and employed an actor‐critic architecture with a centralized critic and decentralized actors to avoid the problem of a non‐stationary environment. Fares et al.…”

Section: Introductionmentioning

confidence: 99%

Cooperative decision making for connected automated vehicles in multiple driving scenarios

Wang

Zhu

et al. 2022

IET Intelligent Trans Sys

View full text Add to dashboard Cite

show abstract

“…Based on these merits, the DRL‐based CAV controllers (S. Chen et al., 2021; Chong et al., 2013; Guan et al., 2019; M. Li et al., 2020; Yipei Wang et al., 2021; M. Zhou et al., 2020) have been favored in recent years, most of which focus on distance tracking and energy efficiency. Furthermore, the multi‐agent reinforcement learning‐based CAV controllers (S. Chen et al., 2021; Ha et al., 2020; Nakka et al., 2021; Shi et al., 2021) can further improve control performance due to its cooperative decision‐making manner.…”

Section: Introductionmentioning

confidence: 99%

“…Zhou et al, 2020) have been favored in recent years, most of which focus on distance tracking and energy efficiency. Furthermore, the multi-agent reinforcement learning-based CAV controllers (S. Ha et al, 2020;Nakka et al, 2021;Shi et al, 2021) can further improve control performance due to its cooperative decision-making manner.…”

mentioning

confidence: 99%

A deep reinforcement learning‐based distributed connected automated vehicle control under communication failure

Shi

Zhou

Wang

et al. 2022

Computer aided Civil Eng

View full text Add to dashboard Cite

This paper proposes a deep reinforcement learning (DRL)-based distributed longitudinal control strategy for connected and automated vehicles (CAVs) under communication failure to stabilize traffic oscillations. Specifically, the signalinterference-plus-noise ratio-based vehicle-to-vehicle communication is incorporated into the DRL training environment to reproduce the realistic communication and time-space varying information flow topologies (IFTs). A dynamic information fusion mechanism is designed to smooth the high-jerk control signal caused by the dynamic IFTs. Based on that, each CAV controlled by the DRL-based agent was developed to receive the real-time downstream CAVs' state information and take longitudinal actions to achieve the equilibrium consensus in the multi-agent system. Simulated experiments are conducted to tune the communication adjustment mechanism and further validate the control performance, oscillation dampening performance and generalization capability of our proposed algorithm. INTRODUCTIONTraffic oscillations, known as the stop-and-go phenomenon (X. Li et al., 2010), contribute to traffic flow instability, traffic unsafety, and energy inefficiency. Connected and automated vehicles (CAVs), equipped with advanced communication and automation capability, have great potential to alleviate traffic oscillations to enhance the traffic flow performance through adaptive cruise control (ACC; Marsden et al., 2001) and cooperative ACC (CACC;Arem et al., 2006).

show abstract

A Control Framework for Socially Optimal Emerging Mobility Systems

Malikopoulos

2024

Springer Tracts on Transportation and Traffic

View full text Add to dashboard Cite

A Multi-Agent Deep Reinforcement Learning Coordination Framework for Connected and Automated Vehicles at Merging Roadways

Cited by 10 publications

References 26 publications

Cooperative decision making for connected automated vehicles in multiple driving scenarios

Cooperative decision making for connected automated vehicles in multiple driving scenarios

A deep reinforcement learning‐based distributed connected automated vehicle control under communication failure

A Control Framework for Socially Optimal Emerging Mobility Systems

Contact Info

Product

Resources

About