A challenging application of artificial intelligence systems involves the scheduling of traffic signals in multi-intersection vehicular networks. This paper introduces a novel use of a multi-agent system and reinforcement learning (RL) framework to obtain an efficient traffic signal control policy. The latter is aimed at minimising the average delay, congestion and likelihood of intersection cross-blocking. A five-intersection traffic network has been studied in which each intersection is governed by an autonomous intelligent agent. Two types of agents, a central agent and an outbound agent, were employed. The outbound agents schedule traffic signals by following the longest-queue-first (LQF) algorithm, which has been proved to guarantee stability and fairness, and collaborate with the central agent by providing it local traffic statistics. The central agent learns a value function driven by its local and neighbours' traffic conditions. The novel methodology proposed here utilises the Q-Learning algorithm with a feedforward neural network for value function approximation. Experimental results clearly demonstrate the advantages of multi-agent RL-based control over LQF governed isolated single-intersection control, thus paving the way for efficient distributed traffic signal control in complex settings.

No abstract

Traffic signal optimization programs have been used widely among transportation professionals. However, none of the existing computer programs can optimize all four traffic control parameters (i.e., cycle length, green split, offset, and phase sequence) simultaneously, even for undersaturated conditions. In this paper, a genetic algorithm-based signal optimization program that can handle oversaturated signalized intersections is presented. The program consists of a genetic algorithm (GA) optimizer and a mesoscopic traffic simulator. The GA optimizer is designed to search for a near-optimal traffic signal timing plan on the basis of a fitness value obtained from the mesoscopic simulator. The proposed program is compared with the newly released TRANSYT-7F version 8.1 on the basis of CORSIM simulation program. Three different demand volume levels-low, medium, and high demand-are tested. For the low-demand and high-demand volume cases, the GAbased program produced statistically better signal timing plans than did TRANSYT-7F in terms of queue time. In the case of mediumdemand volume level, the signal timing plan obtained from the GA-based program produced statistically equivalent queue time compared with TRANSYT-7F. Both programs are judged to provide superior capability for oversaturated conditions due to their queue blockage model when compared with previously available signal timing optimization software.Traffic congestion during peak periods is prevalent for most urban areas. A recent study notes urban arterial systems have experienced increasing traffic congestion (1). Thus, there is a need for effectively managing traffic signal control systems during congested or oversaturated periods. Oversaturated conditions are defined as the condition when vehicles are prevented from moving freely, either because of the presence of vehicles in the intersection itself or because of queue backup in any of the exit links of the intersection (2). Even though oversaturated conditions may last only briefly, the aftereffect may take a long time to clear.Traffic signal coordination and optimization are desirable as costeffective means of reducing urban traffic congestion, especially when additional road construction is impossible because of either high construction cost or lack of available land. Therefore, optimal traffic control plans that would maximize the operational efficiency of existing facilities should be developed and implemented. This can be achieved by maximizing the use of green time and preventing formation of queue blocking of output flows. BACKGROUND Signal OptimizationCurrent traffic signal optimization programs fall into two categories: delay-based models and bandwidth-based models. TRANSYT, a representative delay-based model, minimizes a linear combination of network-wide delay and stops by optimizing cycle length, green split, and offset. In contrast, bandwidth-based programs maximize the sum of directional bands for progression by choosing optimal phase sequence, offset, and cycle length.The limitation of exis...