Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey

Haydari, Ammar; Yılmaz, Yasin

doi:10.48550/arxiv.2005.00935

Cited by 5 publications

(11 citation statements)

References 117 publications

(165 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The agent is trained for multiple episodes where each episode consists of multiple iterations. The cumulative sum of discounted reward under the policy 𝜋 𝜃 at tth iteration of an episode is defined as G t (Equation 3) 24 :…”

Section: Advanced Methodology For Drlmentioning

confidence: 99%

See 1 more Smart Citation

Deep reinforcement learning based cooperative control of traffic signal for multi‐intersection network in intelligent transportation system using edge computing

Paul

2022

Trans Emerging Tel Tech

View full text Add to dashboard Cite

In the current era, the coordination of traffic flow is hindered by the discrepancy between road infrastructure and the number of vehicles which leads to traffic congestion. One of the widely used strategies to mitigate traffic congestion is to control traffic signals with the help of deep reinforcement learning (DRL) in edge computing based intelligent transportation system. This article provides a comprehensive analysis of the most recent DRL algorithms, advantage actor‐critic and proximal policy optimization in multiple deep neural networks (DNNs), including a state‐of‐the‐art transformer model for effective traffic signal management. Here, a single DRL agent is used, which obtains the spatio‐temporal information of the traffic to identify traffic patterns from complex intersection environments. The agent uses this information as the input to the DNNs and then applies the algorithms to retrieve the essential parameters of DNN to seek an optimal action selection policy to mitigate congestion. Different real‐time maps and small city networks are explored here to determine which DNN is best suited for traffic congestion management. The simulation study reveals that both the algorithms significantly outperform the baseline. The transformer model gives the best result when compared to other DNNs. The transformer model decreases average waiting time by 96.16%, implying that it has a higher capability of dealing with congested environments.

show abstract

Section: Advanced Methodology For Drlmentioning

confidence: 99%

“…The update rule by gradient-ascent for 𝜃 at (k + 1)th episode (ie, 𝜃 (k+1) ) is defined in Equation (5). 24 𝜃 (k+1) = 𝜃 k + 𝛼∇J(𝜃 k ).…”

Section: Advanced Methodology For Drlmentioning

confidence: 99%

Deep reinforcement learning based cooperative control of traffic signal for multi‐intersection network in intelligent transportation system using edge computing

Paul

2022

Trans Emerging Tel Tech

View full text Add to dashboard Cite

show abstract

“…RL is the method that can assist intelligent decision-making. It takes sequential actions similar to Markov Decision Process (MDP) with a rewarding criterion (5). It interacts with the environment and try to learn how to behave to maximize rewards, which can be used in complex systems such as a transportation network.…”

Section: Introductionmentioning

confidence: 99%

Lyapunov Function Consistent Adaptive Network Signal Control with Back Pressure and Reinforcement Learning

Ma¹,

Wang²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

This research studies the network traffic signal control problem. It uses the Lyapunov control function to derive the back pressure method, which is equal to differential queue lengths weighted by intersection lane flows. Lyapunov control theory is a platform that unifies several current theories for intersection signal control. We further use the theorem to derive the flow-based and other pressure-based signal control algorithms. For example, the Dynamic, Optimal, Real-time Algorithm for Signals (DORAS) algorithm may be derived by defining the Lyapunov function as the sum of queue length. The study then utilizes the back pressure as a reward in the reinforcement learning (RL) based network signal control, whose agent is trained with double Deep Q-Network (Double-DQN). The proposed algorithm is compared with several traditional and RL-based methods under passenger traffic flow and mixed flow with freight traffic, respectively. The numerical tests are conducted on a single corridor and on a local grid network under three traffic demand scenarios of low, medium, and heavy traffic, respectively. The numerical simulation demonstrates that the proposed algorithm outperforms the others in terms of the average vehicle waiting time on the network.

show abstract

“…This is in part due to many recent MARL innovations utilising novel information structures, such as centralised training schemes to overcome issues of non-stationarity (Lowe et al, 2017) and learned networked communication protocols for scalable and effective cooperation (Foerster et al, 2016;Sukhbaatar et al, 2016;Chu et al, 2020). Moreover, critical infrastructure responsible for society's wellbeing have been demonstrated to be amenable to multi-agent system control and include management of electricity, telecommunications and transportation systems (Herrera et al, 2020;Haydari and Yilmaz, 2020;Chu et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

A game-theoretic analysis of networked system control for common-pool resource management using multi-agent reinforcement learning

Pretorius,

Cameron,

van Biljon

et al. 2020

Preprint

View full text Add to dashboard Cite

Multi-agent reinforcement learning has recently shown great promise as an approach to networked system control. Arguably, one of the most difficult and important tasks for which large scale networked system control is applicable is common-pool resource management. Crucial common-pool resources include arable land, fresh water, wetlands, wildlife, fish stock, forests and the atmosphere, of which proper management is related to some of society's greatest challenges such as food security, inequality and climate change. Here we take inspiration from a recent research program investigating the game-theoretic incentives of humans in social dilemma situations such as the well-known tragedy of the commons. However, instead of focusing on biologically evolved human-like agents, our concern is rather to better understand the learning and operating behaviour of engineered networked systems comprising general-purpose reinforcement learning agents, subject only to nonbiological constraints such as memory, computation and communication bandwidth. Harnessing tools from empirical game-theoretic analysis, we analyse the differences in resulting solution concepts that stem from employing different information structures in the design of networked multi-agent systems. These information structures pertain to the type of information shared between agents as well as the employed communication protocol and network topology. Our analysis contributes new insights into the consequences associated with certain design choices and provides an additional dimension of comparison between systems beyond efficiency, robustness, scalability and mean control performance. ‡

show abstract

Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey

Cited by 5 publications

References 117 publications

Deep reinforcement learning based cooperative control of traffic signal for multi‐intersection network in intelligent transportation system using edge computing

Deep reinforcement learning based cooperative control of traffic signal for multi‐intersection network in intelligent transportation system using edge computing

Lyapunov Function Consistent Adaptive Network Signal Control with Back Pressure and Reinforcement Learning

A game-theoretic analysis of networked system control for common-pool resource management using multi-agent reinforcement learning

Contact Info

Product

Resources

About