Intelligent traffic light control using distributed multi-agent Q learning

Abstract

The combination of Artificial Intelligence (AI) and Internet-of-Things (IoT), which is denoted as AI powered Internet-of-Things (AIoT), is capable of processing huge amount of data generated from large number of devices and handling complex problems in social infrastructures. As AI and IoT technologies are becoming mature, in this paper, we propose to apply AIoT technologies for traffic light control, which is an essential component for intelligent transportation system, to improve the efficiency of smart city's road system. Specifically, various sensors such as surveillance cameras provide real-time information for intelligent traffic light control system to observe the states of both motorized traffic and non-motorized traffic. In this paper, we propose an intelligent traffic light control solution by using distributed multi-agent Q learning, considering the traffic information at the neighboring intersections as well as local motorized and non-motorized traffic, to improve the overall performance of the entire control system. By using the proposed multi-agent Q learning algorithm, our solution is targeting to optimize both the motorized and non-motorized traffic. In addition, we considered many constraints / rules for traffic light control in the real world, and integrate these constraints in the learning algorithm, which can facilitate the proposed solution to be deployed in real operational scenarios. We conducted numerical simulations for a real-world map with realworld traffic data. The simulation results show that our proposed solution outperforms existing solutions in terms of vehicle and pedestrian queue lengths, waiting time at intersections, and many other key performance metrics.

Lightweight PVIDNet: A Priority Vehicles Detection Network Model Based on Deep Learning for Intelligent Traffic Lights

“…However, sensors were used in [63] to signalize traffic lights. In [18], an AI system combined to IoT was implemented; in this system, cameras controlled traffic lights in a city. However, the study focused on pedestrians and not priority vehicles.…”

“…For capturing and analyze traffic images, it is necessary to work with almost real-time processing. Studies about intelligent traffic light commonly use, in addition to sensors, images to detect different types of vehicles, such as emergency vehicles [17,18]. Due to the relevance of waiting time in traffic to emergency vehicles, commonly, studies propose a traffic light that gives priority to these vehicles through both audible sensors and images.…”

Reward Value-Based Goal Selection for Agents’ Cooperative Route Learning Without Communication in Reward and Goal Dynamism

“…As for the results of that, the agents A and B do not reach the optimal goals with both conditions. However, the worst case is that the agents cooperate among the goal located at (4,3) and (5,2) before the environmental change, and (4,2) and (4,3) after that.…”

“…Multi-agent system (MAS) simulates real-world problems by several number of agents which have the own roles in the simulated environment, e.g., robots in a storehouse, traffic lights in an intersection and more [2,4]. To control the agents cooperatively, each agent learns the own action selection policy by reinforcement learning (RL).…”

Evaluation of Reinforcement Learning Traffic Signalling Strategies for Alternative Objectives: Implementation in the Network of Nicosia, Cyprus

“…The evaluated reward mechanisms are namely (a) the vehicle throughput, (b) the average queue length and (c) the total volume of emissions. Although, the environmental effect of RL-based traffic signal control optimisation methodologies has been considered in the past (Liu et al, 2018;Zhu et al, 2015) no other study has evaluated the utilisation of the volume of emissions as the objective value to be optimised.…”