Distributed Cooperative Reinforcement Learning-Based Traffic Signal Control That Integrates V2X Networks’ Dynamic Clustering

Liu, Weirong; Qin, Gaorong; Yun, He; Jiang, Fei

doi:10.1109/tvt.2017.2702388

Cited by 107 publications

(43 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the traffic signal control process, two agents shift behaviors from working and waiting. Through the dual-agent control algorithm, the fixed phasing sequence which is commonly ignored in some DRL methods [11,13] is guaranteed. Due to enough intervals between each control action, the traffic light flicker is avoided, and the system computation load is alleviated.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

“…There are mainly two kinds of traffic signal control algorithms based on DRL to solve the real-time traffic signal control in a four-phase signalized intersection. One is the control model which ignored the fixed phasing sequence of traffic signal in real-life intersections [11][12][13][14]. It defines each phase as an agent action selected according to the maximal Q-value, which changes randomly.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Double Deep Q-Network with a Dual-Agent for Traffic Signal Control

Fang

Sheng

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

Adaptive traffic signal control (ATSC) based on deep reinforcement learning (DRL) has shown promising prospects to reduce traffic congestion. Most existing methods keeping traffic signal phases fixed adopt two agent actions to match a four-phase suffering unstable performance and undesirable operation in a four-phase signalized intersection. In this paper, a Double Deep Q-Network (DDQN) with a dual-agent algorithm is proposed to obtain a stable traffic signal control policy. Specifically, two agents are denoted by two different states and shift the control of green lights to make the phase sequence fixed and control process stable. State representations and reward functions are presented by improving the observability and reducing the leaning difficulty of two agents. To enhance the feasibility and reliability of two agents in the traffic control of the four-phase signalized intersection, a network structure incorporating DDQN is proposed to map states to rewards. Experiments under Simulation of Urban Mobility (SUMO) are carried out, and results show that the proposed traffic signal control algorithm is effective in improving traffic capacity.

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Double Deep Q-Network with a Dual-Agent for Traffic Signal Control

Fang

Sheng

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Liu et al [45] presented a reinforcement learning traffic control method integrating a dynamic clustering algorithm. In this paper, a dynamic clustering algorithm is proposed to achieve a relatively stable cluster structure and enhance traffic data collection efficiency.…”

Section: Control Strategies For Multiple Intersectionsmentioning

confidence: 99%

“…Furthermore, SUMO can couple other network simulation software to achieve V2X environment simulation platform. In [45], a network simulator NS3 and SUMO are connected by the TraCI interface. During the simulation, the VANET nodes in NS3 is used to execute the data and the fixed nodes corresponding to intersections generate the traffic data information and produce adaptive traffic signal control commands.…”

Section: Aimsun Ng [57] Spainmentioning

confidence: 99%

“…This may be an effective method for a traffic signal control system that is hard to build a better mathematical model, especially in a connected vehicle environment. Although [45,61] proposed the reinforcement learning based signal control methods and [32] developed a traffic signal control based on approximate dynamic programming, research on intelligent control in a connected vehicle environment is still limited. Intelligent control will attract more researchers' attention.…”

mentioning

confidence: 99%

See 1 more Smart Citation

An Adaptive Traffic Signal Control in a Connected Vehicle Environment: A Systematic Review

2017

View full text Add to dashboard Cite

Abstract:In the last few years, traffic congestion has become a growing concern due to increasing vehicle ownerships in urban areas. Intersections are one of the major bottlenecks that contribute to urban traffic congestion. Traditional traffic signal control systems cannot adjust the timing pattern depending on road traffic demand. This results in excessive delays for road users. Adaptive traffic signal control in a connected vehicle environment has shown a powerful ability to effectively alleviate urban traffic congestions to achieve desirable objectives (e.g., delay minimization). Connected vehicle technology, as an emerging technology, is a mobile data platform that enables the real-time data exchange among vehicles and between vehicles and infrastructure. Although several reviews about traffic signal control or connected vehicles have been written, a systemic review of adaptive traffic signal control in a connected vehicle environment has not been made. Twenty-six eligible studies searched from six databases constitute the review. A quality evaluation was established based on previous research instruments and applied to the current review. The purpose of this paper is to critically review the existing methods of adaptive traffic signal control in a connected vehicle environment and to compare the advantages or disadvantages of those methods. Further, a systematic framework on connected vehicle based adaptive traffic signal control is summarized to support the future research. Future research is needed to develop more efficient and generic adaptive traffic signal control methods in a connected vehicle environment.

show abstract