Design of Reinforcement Learning Parameters for Seamless Application of Adaptive Traffic Signal Control

Abstract

Adaptive traffic signal control (ATSC) is a promising technique to alleviate traffic congestion. This article focuses on the development of an adaptive traffic signal control system using Reinforcement Learning (RL) as one of the efficient approaches to solve such stochastic closed loop optimal control problem. A generic RL control engine is developed and applied to a multiphase traffic signal at an isolated intersection in Downtown Toronto in a simulation environment. Paramics, a microscopic simulation platform, is used to train and evaluate the adaptive traffic control system. This article investigates the following dimensions of the control problem: 1) RL learning methods, 2) traffic state representations, 3) action selection methods, 4) traffic signal phasing schemes, 5) reward definitions, and 6) variability of flow arrivals to the intersection. The system was tested on three networks (i.e., small, medium, large-scale) to ensure seamless transferability of the system design and results. The RL controller is benchmarked against optimized pretimed control and actuated control. The RL-based controller saves 48% average vehicle delay when compared to optimized pretimed controller and fully-actuated controller. In addition, the effect of the best design of RL-based ATSC system is tested on a large-scale application of 59 intersections in downtown Toronto and the results are compared versus the base case scenario of signal control systems in the field which are mix of pretimed and actuated controllers. The RL-based ATSC results in the following savings: average delay (27%), queue length (28%), and l CO 2 emission factors (28%).

Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures

“…We also experimented with other reward functions from the literature (see [6]) during our preliminary experiments; we consistently found the reward in Eq. 3 to be the best.…”

Evaluating Reinforcement Learning State Representations for Adaptive Traffic Signal Control

“…The most common reward definitions are change in delay [15] [8] and change in queued vehicles [6][13] [14]. For a comprehensive review of reinforcement learning traffic signal control research, the reader is referred to [16] and [17].…”

Assessment of self-learning adaptive traffic signal control on congested urban areas: independent versus coordinated perspectives

“…Below is a brief description of each of the design parameters and the best selected parameters based on the findings of El-Tantawy and Abdulhai (2011).…”