Traffic Light Cycle Configuration of Single Intersection Based on Modified Q-Learning

Abstract

In recent years, within large cities with a high population density, traffic congestion has become more and more serious, resulting in increased emissions of vehicles and reducing the efficiency of urban operations. Many factors have caused traffic congestion, such as insufficient road capacity, high vehicle density, poor urban traffic planning and inconsistent traffic light cycle configuration. Among these factors, the problems of traffic light cycle configuration are the focal points of this paper. If traffic lights can adjust the cycle dynamically with traffic data, it will reduce degrees of traffic congestion significantly. Therefore, a modified mechanism based on Q-Learning to optimize traffic light cycle configuration is proposed to obtain lower average vehicle delay time, while keeping significantly fewer processing steps. The experimental results will show that the number of processing steps of this proposed mechanism is 11.76 times fewer than that of the exhaustive search scheme, and also that the average vehicle delay is only slightly lower than that of the exhaustive search scheme by 5.4%. Therefore the proposed modified Q-learning mechanism will be capable of reducing the degrees of traffic congestions effectively by minimizing processing steps.

Design of Deep Reinforcement Learning Approach for Traffic Signal Control at Three-way Crossroads

“…Xiem HV et al Learning only according to data availability and computational resources, complex parameter tuning, and model optimization, not always meet the continuously evolving traffic requirements Chao et al (2008) [26] Recognizing the traffic flow based on extension neural network [33] A modified Q-learning mechanism to find the optimal cycle configuration Kumar et al (2019) [32] The traffic signal phase and duration are dynamically adjusted using Deep Q-learning Bouktif et al (2021) [34] Combining both discrete and continuous DRL approaches A key advantage is the agent's ability to learn and improve without supervision or prior knowledge of the specific scene. The agent takes actions (actiona t ) that directly influence the environment and receives status information (states t ) from each field, along with a reward (rewardr t ) that reflects its performance.…”

“…These networks were utilized to generate Q-values as outputs. Other studies have focused on dynamic traffic control by adjusting traffic light cycles based on real-time traffic information using modified Q-learning mechanisms or deep reinforcement learning models [32,33]. Wei et al [4] have also developed intelligent traffic light systems that employ reinforcement learning to replace conventional manually designed rules and reduce average waiting time.…”

Traffic Control Based on Integrated Kalman Filtering and Adaptive Quantized Q-Learning Framework for Internet of Vehicles

“…Several challenges occur in modern cities such as vehicle emissions caused by the transportation systems in general and traffic control [9]. Effective congestion control requires a combination of various technologies, including fast sensor reading and processing, vehicles to vehicles (V2V) and vehicles to infrastructure (V2I) communication, data integration, and intelligent deep learning techniques [10,11]. These technologies can be used to improve congestion control by enabling real-time monitoring and management of traffic flow.…”

Application-Based Online Traffic Classification with Deep Learning Models on SDN Networks

“…These processes will significantly reduce the efficiency of network traffic processing and consequently increase the overall delay and processing loading. Furthermore, the development of software defined network (SDN) has made network management more effective and efficient [1][2]. Unlike traditional networks, SDN separates the network layer into a data plane and a control plane which could be a logical control unit and programmable.…”