Multi-Agent Reinforcement Learning for Cooperative Adaptive Cruise Control

Peake, Ashley; McCalmon, Joe; Raiford, Benjamin; Liu, Tongtong; Alqahtani, Sarra

doi:10.1109/ictai50040.2020.00013

Cited by 26 publications

(21 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to improve the limitations of vehicle following models, DRL has been a steady area of research in the AV community, with many authors contributing works to DRL applied to CACC [8,9,22,23] . In a study by Lin et al, a DRL framework is designed to control a CACC AV platoon [22] .…”

Section: Deep Reinforcement Learning Applied To Av Platooningmentioning

confidence: 99%

“…The DRL framework uses the deep deterministic policy gradient (DDPG) [24] algorithm and is found to have near-optimal performance [22] . In addition, Peake et al identify limitations in platooning with regard to the communication in platooning [23] . Through the application of a multi-agent reinforcement learning process, i.e.…”

Section: Deep Reinforcement Learning Applied To Av Platooningmentioning

confidence: 99%

“…Through the application of a multi-agent reinforcement learning process, i.e. a policy gradient RL and LSTM network, the performance of a platoon containing 3-5 vehicles is improved upon that of current RL applications to platooning [23] . Furthermore, Model Predictive Control (MPC) is the current state-of-the-art for real-time optimal control practices [25] .…”

Section: Deep Reinforcement Learning Applied To Av Platooningmentioning

confidence: 99%

“…Many of the works existing on FRL have shown the benefits of FRL with regard to the increased rate of convergence and overall system performance with distributed networks, edge caching and communications [16][17][18][19] . Furthermore, of the works cited in this study, the works closely related to FRL for platoon control are those of Peake et al and Liang et al [15,23] . In contrast to Liang et al, where FedAvg is applied successfully to control the steering angle of a single vehicle, we apply FRL to an AV platooning problem where the control of multiple vehicles' positions and spacing are required [15] .…”

Section: Contributionsmentioning

confidence: 99%

“…In contrast to Liang et al, where FedAvg is applied successfully to control the steering angle of a single vehicle, we apply FRL to an AV platooning problem where the control of multiple vehicles' positions and spacing are required [15] . Peake et al explore multi-agent reinforcement learning and its ability to improve the performance of AV platoons experiencing communication delays [23] . Although Peake et al are also successful in their approach, there is no specific reference to FRL in the paper [23] .…”

Section: Contributionsmentioning

confidence: 99%

See 4 more Smart Citations

AVDDPG – Federated reinforcement learning applied to autonomous platoon control

Boin¹,

Lei²,

Yang³

2022

Intell Robot

View full text Add to dashboard Cite

Since 2016 federated learning (FL) has been an evolving topic of discussion in the artificial intelligence (AI) research community. Applications of FL led to the development and study of federated reinforcement learning (FRL). Few works exist on the topic of FRL applied to autonomous vehicle (AV) platoons. In addition, most FRL works choose a single aggregation method (usually weight or gradient aggregation). We explore FRL's effectiveness as a means to improve AV platooning by designing and implementing an FRL framework atop a custom AV platoon environment. The application of FRL in AV platooning is studied under two scenarios: (1) Inter-platoon FRL (Inter-FRL) where FRL is applied to AVs across different platoons; (2) Intra-platoon FRL (Intra-FRL) where FRL is applied to AVs within a single platoon. Both Inter-FRL and Intra-FRL are applied to a custom AV platooning environment using both gradient and weight aggregation to observe the performance effects FRL can have on AV platoons relative to an AV platooning environment trained without FRL. It is concluded that Intra-FRL using weight aggregation (Intra-FRLWA) provides the best performance for controlling an AV platoon. In addition, we found that weight aggregation in FRL for AV platooning provides increases in performance relative to gradient aggregation. Finally, a performance analysis is conducted for Intra-FRLWA versus a platooning environment without FRL for platoons of length 3, 4 and 5 vehicles. It is concluded that Intra-FRLWA largely out-performs the platooning environment that is trained without FRL.

show abstract