Traffic Light Control Using Hierarchical Reinforcement Learning and Options Framework

Abstract

The number of vehicles worldwide has grown rapidly over the past decade, impacting how urban traffic is managed. Traffic light control is a well-known problem and, although an increasing number of technologies are used to solve it, it still poses challenges and opportunities, especially when considering the inefficiency of the popular fixed-time traffic controllers. This study aims to apply Hierarchical Reinforcement Learning (HRL) and Options Framework to control a signalized vehicular intersection and compare its performance with that of a fixed-time traffic controller, configured using the Webster Method. HRL combines the ability to learn and make decisions while taking observations from the environment in realtime. These capabilities bring a significant adaptive power to a highly dynamic problem. The test scenarios were built using the SUMO simulation tool. According to our results, HRL presents better performance than those of its own isolated sub-policies and the fixed-time model, indicating a simple and efficient alternative.

Intelligent Traffic Light Control System with Priority Lane Intervention

“…Borges et al [35] proposed traffic light control using Hierarchical Reinforcement Learning (HRL) and an Options Framework. HRL is used to establish a sub-policy to maximize vehicle flow and minimize waiting time.…”

“…Several studies were conducted to obtain the optimal traffic light signal time interval. Literatures [28][29][30][31][32][33][34][35] simulated traffic light control with different platforms and methods. Meanwhile, [36] develop a prototype of a traffic light signal in pedestrian crossing areas.…”

“…Miao and Leitner [34] Simulation: SUMO Develop a traffic light signal scheduling as service information on Connected and Automated Vehicles (CAV). Borges et al [35] Simulation: SUMO Controlling traffic lights using Hierarchical Reinforcement Learning (HRL) and Options Framework. Saad et al [36] Prototipe: Arduino microcontroller and Passive Infrared (PIR) sensors…”