Traffic Signal Control with Successor Feature-Based Deep Reinforcement Learning Agent

Abstract

In this paper, we study the problem of traffic signal control in general intersections by applying a recent reinforcement learning technique. Nowadays, traffic congestion and road usage are increasing significantly as more and more vehicles enter the same infrastructures. New solutions are needed to minimize travel times or maximize the network capacity (throughput). Recent studies embrace machine learning approaches that have the power to aid and optimize the increasing demands. However, most reinforcement learning algorithms fail to be adaptive regarding goal functions. To this end, we provide a novel successor feature-based solution to control a single intersection to optimize the traffic flow, reduce the environmental impact, and promote sustainability. Our method allows for flexibility and adaptability to changing circumstances and goals. It supports changes in preferences during inference, so the behavior of the trained agent (traffic signal controller) can be changed rapidly during the inference time. By introducing the successor features to the domain, we define the basics of successor features, the base reward functions, and the goal preferences of the traffic signal control system. As our main direction, we tackle environmental impact reduction and support prioritized vehicles’ commutes. We include an evaluation of how our method achieves a more effective operation considering the environmental impact and how adaptive it is compared to a general Deep-Q-Network solution. Aside from this, standard rule-based and adaptive signal-controlling technologies are compared to our method to show its advances. Furthermore, we perform an ablation analysis on the adaptivity of the agent and demonstrate a consistent level of performance under similar circumstances.

WITHDRAWN: Regional Traffic Signal Control Optimization Model based on Quantum Genetic Algorithm

“…Since the 1910s, with the development of the heuristic algo rithm, it has become an algorithm for solving combinatorial optimization problems, where each instance has a feasible solution. The heuristic algorithm has been widely applied in solving intelligent traffic control problems due to its distinctive traits and capabilities [12][13][14][15][16][17][18][19].…”

A Review of Deep Reinforcement Learning for Traffic Signal Control

“…In traffic signal control systems, DRL-based approaches and algorithms have resulted in excellent results. For example, the DRL algorithm with experience replay and target network [5], Adaptive traffic signal control system (ATCS) [17], Successor feature [33], and delay-based fairness and throughput-based fairness [35]. However, these methods and algorithms are more effective than fixedtime traffic light control.…”

Una revisión sistemática de literatura sobre implementaciones de sistemas de control de tráfico

“…Es un método que utiliza redes neuronales recurrentes para el procesamiento de datos secuenciales, como el procesamiento del lenguaje natural y la traducción automática. (Damadam et al, 2022); (Korecki & Helbing, 2022); ; (Yuan et al, 2021); (Wakkumbura et al, 2021); (Szoke et al, 2023); (Impedovo et al, 2019) Machine Learning…”

“…Los autores señalan que mientras cuente con buen entrenamiento aprender una política efectiva puede ser una de las mejores opciones para el control de tráfico debido a su procesamiento por redes. (Damadam et al, 2022); (Rasheed et al, 2022); (Tan et al, 2022); (Wu et al, 2022); (Wang et al, 2023); (Li, Z. et al, 2021); (Szoke et al, 2023);…”