A deep reinforcement learning-based cooperative approach for multi-intersection traffic signal control

Haddad, Tarek Amine; Hedjazi, Djalal; Aouag, Sofiane

doi:10.1016/j.engappai.2022.105019

Cited by 31 publications

(12 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite its advantages, RL faces challenges in solving TSC problems. Centralized learning models can achieve optimal global results but require excessive communication and computational resources 40 . Distributed learning solutions allocate agents to learn local models, hindering generalization across different intersections 18 .…”

Section: Related Workmentioning

confidence: 99%

A scalable approach to optimize traffic signal control with federated reinforcement learning

Bao,

Wu,

Lin

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Intelligent Transportation has seen significant advancements with Deep Learning and the Internet of Things, making Traffic Signal Control (TSC) research crucial for reducing congestion, travel time, emissions, and energy consumption. Reinforcement Learning (RL) has emerged as the primary method for TSC, but centralized learning poses communication and computing challenges, while distributed learning struggles to adapt across intersections. This paper presents a novel approach using Federated Learning (FL)-based RL for TSC. FL integrates knowledge from local agents into a global model, overcoming intersection variations with a unified agent state structure. To endow the model with the capacity to globally represent the TSC task while preserving the distinctive feature information inherent to each intersection, a segment of the RL neural network is aggregated to the cloud, and the remaining layers undergo fine-tuning upon convergence of the model training process. Extensive experiments demonstrate reduced queuing and waiting times globally, and the successful scalability of the proposed model is validated on a real-world traffic network in Monaco, showing its potential for new intersections.

show abstract

Section: Related Workmentioning

confidence: 99%

A scalable approach to optimize traffic signal control with federated reinforcement learning

Bao,

Wu,

Lin

et al. 2023

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Despite its advantages, RL faces challenges in solving TSC problems. Centralized learning models can achieve optimal global results but require excessive communication and computational resources 29 . Distributed learning solutions allocate agents to learn local models, hindering generalization across different intersections 30 .…”

Section: Related Workmentioning

confidence: 99%

Optimizing Traffic Signal Control: A Scalable Approach with Federated Reinforcement Learning

Bao,

Wu,

Lin

et al. 2023

Preprint

View full text Add to dashboard Cite

Intelligent Transportation has seen significant advancements with Deep Learning (DL) and Internet of Things, making Traffic Signal Control (TSC) research crucial for reducing congestion, travel time, emissions, and energy consumption. Reinforcement Learning (RL) has emerged as the primary method for TSC, but centralized learning poses communication and computing challenges, while distributed learning struggles to adapt across intersections. This paper presents a novel approach using Federated Learning (FL)-based RL for TSC. FL integrates knowledge from local agents into a global model, overcoming intersection variations with a unified agent state structure. Additionally, the output layer's parameters are not aggregated to handle different intersection settings; instead, fine-tuning is performed after model training for deployment. Extensive experiments demonstrate reduced queuing and waiting times globally, and successful scalability of the proposed model is validated on a real-world traffic network in Monaco, showing its potential for new intersections.

show abstract

“…While its adaptability is novel, its applicability in interconnected multi-intersection settings is yet to be tested and the algorithm's efficiency needs to be improved. While optimizing the efficiency of intersection traffic flow, some scholars have also taken the reduction in carbon dioxide emissions within the intersection area as an evaluation criterion for optimizing intersection signal control [26][27][28]. Research has shown that intersection signal optimization schemes based on deep reinforcement learning have a certain effect on reducing CO 2 emissions.…”

Section: Introductionmentioning

confidence: 99%

Carbon Dioxide Emission Reduction-Oriented Optimal Control of Traffic Signals in Mixed Traffic Flow Based on Deep Reinforcement Learning

Wang,

Xu,

2023

Sustainability

View full text Add to dashboard Cite

To alleviate intersection traffic congestion and reduce carbon emissions at intersections, research on exploiting reinforcement learning for intersection signal control has become a frontier topic in the field of intelligent transportation. This study utilizes a deep reinforcement learning algorithm based on the D3QN (dueling double deep Q network) to achieve adaptive control of signal timings. Under a mixed traffic environment with connected and automated vehicles (CAVs) and human-driven vehicles (HDVs), this study constructs a reward function (Reward—CO2 Reduction) to minimize vehicle waiting time and carbon dioxide emissions at the intersection. Additionally, to account for the spatiotemporal distribution characteristics of traffic flow, an adaptive-phase action space and a fixed-phase action space are designed to optimize action selections. The proposed algorithm is validated in a SUMO simulation with different traffic volumes and CAV penetration rates. The experimental results are compared with other control strategies like Webster’s method (fixed-time control). The analysis shows that the proposed model can effectively reduce carbon dioxide emissions when the traffic volume is low or medium. As the penetration rate of CAVs increases, the average carbon dioxide emissions and waiting time can be further reduced with the proposed model. The significance of this study lies in its dual achievement: by presenting a flexible strategy that not only reduces the environmental impact by lowering carbon dioxide emissions but also enhances traffic efficiency, it provides a tangible example of the advancement of green intelligent transportation systems.

show abstract

A deep reinforcement learning-based cooperative approach for multi-intersection traffic signal control

Cited by 31 publications

References 30 publications

A scalable approach to optimize traffic signal control with federated reinforcement learning

A scalable approach to optimize traffic signal control with federated reinforcement learning

Optimizing Traffic Signal Control: A Scalable Approach with Federated Reinforcement Learning

Carbon Dioxide Emission Reduction-Oriented Optimal Control of Traffic Signals in Mixed Traffic Flow Based on Deep Reinforcement Learning

Contact Info

Product

Resources

About