CVLight: Decentralized learning for adaptive traffic signal control with connected vehicles

Li, Wangzhi; Zhao, Mobin; Fu, Yongjie; Ruan, Kangrui; Di, Xuan

doi:10.1016/j.trc.2022.103728

Cited by 30 publications

(8 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is also a review of reinforcement-based TSCS [29]. For example, Mo et al's [30], Maadi et al [31], and Chen's [32] research papers are well received. Due to the limitation of our scope of work and the word limitations, we did not conduct a full review of the approaches.…”

Section: Traffic Signal Control Systems At V2x Intersectionsmentioning

confidence: 99%

Distributed Traffic Signal Optimization at V2X Intersections

Zhang,

Zhang

2024

Mathematics

View full text Add to dashboard Cite

This paper presents our research on a traffic signal control system (TSCS) at V2X intersections. The overall objective of the study is to create an implementable TSCS. The specific objective of this paper is to investigate a distributed system towards implementation. The objective function of minimizing queue delay is formulated as the integral of queue lengths. The discrete queueing estimation is mixed with macro and micro traffic flow models. The novel proposed architecture alleviates the communication network bandwidth constraint by processing BSMs and computing queue lengths at the local intersection. In addition, a two-stage distributed system is designed to optimize offsets, splits, and cycle length simultaneously and in real time. The paper advances TSCS theories by contributing a novel analytic formulation of delay functions and their first degree of derivatives for a two-stage optimization model. The open-source traffic simulation engine Enhanced Transportation Flow Open-Source Microscopic Model (ETFOMM version 1.2) was selected as a simulation environment to develop, debug, and evaluate the models and the system. The control delay of the major direction, minor direction, and the total network were collected to assess the system performance. Compared with the optimized TSCS timing plan by the Virginia Department of Transportation, the system generated a 21% control delay reduction in the major direction and a 7% control delay reduction in the minor direction at just a 10% penetration rate of connected vehicles. Finally, the proposed distributed and centralized systems present similar performances in the case study.

show abstract

Section: Traffic Signal Control Systems At V2x Intersectionsmentioning

confidence: 99%

Distributed Traffic Signal Optimization at V2X Intersections

Zhang,

Zhang

2024

Mathematics

View full text Add to dashboard Cite

show abstract

“…The centralized training with decentralized execution framework empowers the scheme to be compatible with stochastic traffic settings and realize negligible communication and synchronization cost among vehicles. Under the actor-critic framework, Mo et al [95] investigated the traffic signal control strategy based on connected vehicle communications. We recommend [96,97] for a comprehensive and systematic review on RL-based traffic signal control methods.…”

Section: Literature Reviewmentioning

confidence: 99%

EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System

Su,

Zhong,

Chow

et al. 2022

Preprint

View full text Add to dashboard Cite

Emergency vehicles (EMVs) play a crucial role in responding to time-critical calls such as medical emergencies and fire outbreaks in urban areas. Existing methods for EMV dispatch typically optimize routes based on historical traffic-flow data and design traffic signal pre-emption accordingly; however, we still lack a systematic methodology to address the coupling between EMV routing and traffic signal control. In this paper, we propose EMVLight, a decentralized reinforcement learning (RL) framework for joint dynamic EMV routing and traffic signal pre-emption. We adopt the multi-agent advantage actor-critic method with policy sharing and spatial discounted factor. This framework addresses the coupling between EMV navigation and traffic signal control via an innovative design of multi-class RL agents and a novel pressure-based reward function. The proposed methodology enables EMVLight to learn network-level cooperative traffic signal phasing strategies that not only reduce EMV travel time but also shortens the travel time of non-EMVs. Simulation-based experiments indicate that EMVLight enables up to a 42.6% reduction in EMV travel time as well as an 23.5% shorter average travel time compared with existing approaches.

show abstract

“…Traffic simulation tools, e.g., Vissim, SUMO, CityFlow, and Aimsun, were widely used to build virtual intersections. Vehicle demand level during training was unchangeable over time , or varied by time of day [13][14][15][16][17][18][19][33][34][35][36][37][38][39][40][41]. Vehicle turning ratios were typically fixed.…”

Section: Training Environmentsmentioning

confidence: 99%

“…To choose a vehicle phase to display green with a specific duration 18 [9,[11][12][13]15,21,[26][27][28][29][31][32][33][34]36,37,39,40] To choose the green time for current vehicle phase 4 [10,17,23,41] To determine whether or not to end current vehicle phase 8 [7,16,[18][19][20]24,25,38] To adjust the green time for all vehicle phases in next cycle 5 [8,14,22,30,35] Vehicle-specific performance measure used to construct rewards Number of already served vehicles 14 [12,13,17,18,[20][21][22][23]28,31,33,34,38,39] Wait time of already ...…”

Section: Action Taken By An Agentmentioning

confidence: 99%

Double Deep Q-Network with Dynamic Bootstrapping for Real-Time Isolated Signal Control: A Traffic Engineering Perspective

Zheng

Chen

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

Real-time isolated signal control (RISC) at an intersection is of interest in the field of traffic engineering. Energizing RISC with reinforcement learning (RL) is feasible and necessary. Previous studies paid less attention to traffic engineering considerations and under-utilized traffic expertise to construct RL tasks. This study profiles the single-ring RISC problem from the perspective of traffic engineers, and improves a prevailing RL method for solving it. By qualitative applicability analysis, we choose double deep Q-network (DDQN) as the basic method. A single agent is deployed for an intersection. Reward is defined with vehicle departures to properly encourage and punish the agent’s behavior. The action is to determine the remaining green time for the current vehicle phase. State is represented in a grid-based mode. To update action values in time-varying environments, we present a temporal-difference algorithm TD(Dyn) to perform dynamic bootstrapping with the variable interval between actions selected. To accelerate training, we propose a data augmentation based on intersection symmetry. Our improved DDQN, termed D3ynQN, is subject to the signal timing constraints in engineering. The experiments at a close-to-reality intersection indicate that, by means of D3ynQN and non-delay-based reward, the agent acquires useful knowledge to significantly outperform a fully-actuated control technique in reducing average vehicle delay.

show abstract

CVLight: Decentralized learning for adaptive traffic signal control with connected vehicles

Cited by 30 publications

References 34 publications

Distributed Traffic Signal Optimization at V2X Intersections

Distributed Traffic Signal Optimization at V2X Intersections

EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System

Double Deep Q-Network with Dynamic Bootstrapping for Real-Time Isolated Signal Control: A Traffic Engineering Perspective

Contact Info

Product

Resources

About