Real-Time Adaptive Traffic Signal Control in a Connected and Automated Vehicle Environment: Optimisation of Signal Planning with Reinforcement Learning under Vehicle Speed Guidance

Abstract

Adaptive traffic signal control (ATSC) is an effective method to reduce traffic congestion in modern urban areas. Many studies adopted various approaches to adjust traffic signal plans according to real-time traffic in response to demand fluctuations to improve urban network performance (e.g., minimise delay). Recently, learning-based methods such as reinforcement learning (RL) have achieved promising results in signal plan optimisation. However, adopting these self-learning techniques in future traffic environments in the presence of connected and automated vehicles (CAVs) remains largely an open challenge. This study develops a real-time RL-based adaptive traffic signal control that optimises a signal plan to minimise the total queue length while allowing the CAVs to adjust their speed based on a fixed timing strategy to decrease total stop delays. The highlight of this work is combining a speed guidance system with a reinforcement learning-based traffic signal control. Two different performance measures are implemented to minimise total queue length and total stop delays. Results indicate that the proposed method outperforms a fixed timing plan (with optimal speed advisory in a CAV environment) and traditional actuated control, in terms of average stop delay of vehicle and queue length, particularly under saturated and oversaturated conditions.

Enhancing Traffic Sustainability: An Analysis of Isolation Intersection Effectiveness through Fixed Time and Logic Control Design Using VisVAP Algorithm

“…The simulation results indicated that travel times and delays were reduced with the application of the Adaptive Traffic Signal Control in all considered time periods, as reported in the study [31]. In his study, Dabiri et al [32], proposes a cost-effective approach to reduce delays in semi-actuated coordinated signal operation without incurring additional costs for installing new detectors or developing adaptive controller systems.…”

“…Common indicators include average queue length and average delays, expressed numerically or percentages. The queue length ratio is calculated by dividing the average queue length (q i ) by its maximum value (Q max ), as defined in Equation ( 21) [32].…”

“…Vehicle delay ratio, is calculated over the duration of the cycle length. Consequently, it's derived by dividing the average delay (d i ) by the cycle length (C), and it's calculated using Equation ( 22) [32].…”

Formulation and Performance Assessment for Multiple Server Queueing Models

“…This research also showed the evaluation and prediction of waiting time and queue length through artificial intelligence as their research gap. Recently, Maadi et al, (2022) adopted a reinforcement learning-based control system to reduce stop delay and queue length in a traffic signal. By training the traffic signal with this framework, automated and connected vehicles are provided with dynamic speed direction, which improves the interaction between the traffic controller and the driver.…”

“…Arrival rate of entity (𝜆) and service rate of server (𝜇) are the input variables to mathematically estimate waiting time, queue length, entity number in the queue and the system (Siddiqui et al, 2020). The queuing theory has many useful applications, including traffic flow (Zhao et al, 2019), (Hurtado Lange and Maguluri, 2022), (Maadi et al, 2022), plant location design (vehicle gas charging stations (Said and Mouftah, 2020), (Xiao et al, 2020)), scheduling (jobs on machine, patients in hospitals, online order taking and computer programs), and facility design (post offices, banks, supermarkets, ticket counter). Using queuing theory, a company can develop more efficient processes, systems, pricing mechanisms, staffing solutions, and arrival management strategies to minimize customer's waiting time and maximize the number of customers served.…”