Reinforcement learning in urban network traffic signal control: A systematic literature review

Noaeen, Mohammad; Naik, Atharva; Goodman, Liana; Crebo, Jared; Abrar, Taimoor; Abad, Zahra Shakeri Hossein; Bazzan, Ana L. C.; Far, Behrouz H.

doi:10.1016/j.eswa.2022.116830

Cited by 113 publications

(34 citation statements)

References 105 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, Rakelly et al (2019) developed a policy-based meta-RL model, and their action-value and policy functions were alternately updated based on off-policy data. The present study accepted the concept of this paper for traffic signal control on an areawide scale, but a value-based model was used instead of a policy-based model, since the former is more suitable for the traffic signal control on a large scale (Noaeen et al, 2022).…”

Section: Related Workmentioning

confidence: 80%

A meta–reinforcement learning algorithm for traffic signal control to automatically switch different reward functions according to the saturation level of traffic flows

Kim

Kang

Sohn

2022

Computer aided Civil Eng

View full text Add to dashboard Cite

Reinforcement learning (RL) algorithms have been widely applied in solving traffic signal control problems. Traffic environments, however, are intrinsically nonstationary, which creates a convergence problem that RL algorithms struggle to overcome. Basically, as a target problem for an RL algorithm, the Markov decision process (MDP) can be solved only when both the transition and reward functions do not vary. Unfortunately, the environment for traffic signal control is not stationary since the goal of traffic signal control varies according to congestion levels. For unsaturated traffic conditions, the objective of traffic signal control should be to minimize vehicle delay. On the other hand, the objective must be to maximize the throughput when traffic flow is saturated. A multiregime analysis is possible for varying conditions, but classifying the traffic regime creates another complex task. The present study provides a meta‐RL algorithm that embeds a latent vector to recognize the different contexts of an environment in order to automatically classify traffic regimes and apply a customized reward for each context. In simulation experiments, the proposed meta‐RL algorithm succeeded in differentiating rewards according to the saturation level of traffic conditions.

show abstract

Section: Related Workmentioning

confidence: 80%

A meta–reinforcement learning algorithm for traffic signal control to automatically switch different reward functions according to the saturation level of traffic flows

Kim

Kang

Sohn

2022

Computer aided Civil Eng

View full text Add to dashboard Cite

show abstract

“…Real-time The algorithm provides near-optimal solutions with a maximum optimality gap of 5.4%. [19] Decentralized Reinforcement Learning at the Edge for traffic light control in the IoV (DRLE)…”

Section: Discussionmentioning

confidence: 99%

A Bibliometric Review and Analysis of Traffic Lights Optimization

Witeck

Rocha

Silva

et al. 2022

Computational Science and Its Applications – ICCSA 2022 Workshops

View full text Add to dashboard Cite

The significant increase in the number of vehicles in urban areas emerges the challenge of urban mobility. Researchers in this area suggest that most daily delays in urban travel times are caused by intersections, which could be reduced if the traffic lights at these intersections were more efficient. The use of simulation for real intersections can be effective in optimizing the cycle times and improving the traffic light timing to coordinate vehicles passing through intersections. From these themes emerge the research questions: How are the existing approaches (optimization techniques and simulation) to managing traffic lights smartly? What kind of data (offline and online) are used for traffic lights optimization? How beneficial is it to propose an optimization approach to the traffic system? This paper aims to answer these questions, carried out through a bibliometric literature review. In total, 93 articles were analyzed. The main findings revealed that the United States and China are the countries with the most studies published in the last ten years. Moreover, Particle Swarm Optimization is a frequently used approach, and there is a tendency for studies to perform optimization of real cases by real-time data, showing that the praxis of smart cities has resorted to smart traffic lights.

show abstract

“…The DRL-based traffic control methods can be classified into two groups according to the traffic control mode: centralized control and decentralized control [20]. The centralized control methods usually have one RL agent which learns the optimal policy for the whole grid.…”

Section: Traffic Control Based On Rlmentioning

confidence: 99%

CenLight: Centralized traffic grid signal optimization via action and state decomposition

Ran

Lou

et al. 2023

IET Intelligent Trans Sys

View full text Add to dashboard Cite

The centralized traffic grid signal control by the reinforcement learning method is challenging due to the difficulties of searching policy in the large state and action space. In order to solve these problems, a deep reinforcement learning (DRL) method via the action and state decomposition mechanism is proposed. We apply long short-term memory to construct the agent which decomposes the high-dimensional state and action space into sub-spaces and makes decisions incrementally. This is a significant difference between our method and other methods. Through the specifically designed structure of the agent, the difficulty of searching policies can be mitigated, and our method can effectively control the traffic lights in a grid with hundreds of intersections. Experiments on synthetic data and real-world data show that our method has better performance than traditional control methods and state-of-the-art DRL-based methods with an improvement of 21% on the queue length on synthetic data and the best travel time with an improvement of 9.35% on real-world data.

show abstract

Reinforcement learning in urban network traffic signal control: A systematic literature review

Cited by 113 publications

References 105 publications

A meta–reinforcement learning algorithm for traffic signal control to automatically switch different reward functions according to the saturation level of traffic flows

A meta–reinforcement learning algorithm for traffic signal control to automatically switch different reward functions according to the saturation level of traffic flows

A Bibliometric Review and Analysis of Traffic Lights Optimization

CenLight: Centralized traffic grid signal optimization via action and state decomposition

Contact Info

Product

Resources

About