Graph neural network and reinforcement learning for multi‐agent cooperative control of connected autonomous vehicles

Abstract

A connected autonomous vehicle (CAV) network can be defined as a set of connected vehicles including CAVs that operate on a specific spatial scope that may be a road network, corridor, or segment. The spatial scope constitutes an environment where traffic information is shared and instructions are issued for controlling the CAVs movements. Within such a spatial scope, high‐level cooperation among CAVs fostered by joint planning and control of their movements can greatly enhance the safety and mobility performance of their operations. Unfortunately, the highly combinatory and volatile nature of CAV networks due to the dynamic number of agents (vehicles) and the fast‐growing joint action space associated with multi‐agent driving tasks pose difficultly in achieving cooperative control. The problem is NP‐hard and cannot be efficiently resolved using rule‐based control techniques. Also, there is a great deal of information in the literature regarding sensing technologies and control logic in CAV operations but relatively little information on the integration of information from collaborative sensing and connectivity sources. Therefore, we present a novel deep reinforcement learning‐based algorithm that combines graphic convolution neural network with deep Q‐network to form an innovative graphic convolution Q network that serves as the information fusion module and decision processor. In this study, the spatial scope we consider for the CAV network is a multi‐lane road corridor. We demonstrate the proposed control algorithm using the application context of freeway lane‐changing at the approaches to an exit ramp. For purposes of comparison, the proposed model is evaluated vis‐à‐vis traditional rule‐based and long short‐term memory‐based fusion models. The results suggest that the proposed model is capable of aggregating information received from sensing and connectivity sources and prescribing efficient operative lane‐change decisions for multiple CAVs, in a manner that enhances safety and mobility. That way, the operational intentions of individual CAVs can be fulfilled even in partially observed and highly dynamic mixed traffic streams. The paper presents experimental evidence to demonstrate that the proposed algorithm can significantly enhance CAV operations. The proposed algorithm can be deployed at roadside units or cloud platforms or other centralized control facilities.

Random fields representation over manifolds via isometric feature mapping‐based dimension reduction

“…The present paper focuses only on the random field representation over complex geometries, while the detailed applications in several fields of uncertainty quantification need more further study. Especially, newer and more powerful supervised ML methods have been developed recently, for example, neural dynamic classification algorithm (Rafiei & Adeli, 2017b), dynamic ensemble learning algorithm (Alam et al, 2019), deep reinforcement learning (Chen et al, 2021;Wang et al, 2021), finite element machine for fast learning (Pereira et al, 2019), etc., which enable us to conveniently and rapidly predict the structural behaviors. These methods have already been successfully applied to material properties prediction (Rafiei et al, 2017;Valikhani et al, 2021), hazard early warning (Dong et al, 2021;Rafiei & Adeli, 2017a), construction cost estimation (Rafiei & Adeli, 2018), risk assessment (Tomar & Burton, 2021), etc.…”

Applications of Multi-Agent Deep Reinforcement Learning: Models and Algorithms

“…In [75], Chen et al embedded SADRL in a centralized agent, which is the roadside units (RUs), to control the mobility of distributed agents, which are autonomous vehicles connected to the RUs, based on traffic conditions in order to manage the congestion level in vehicular networks (A.2). The agents maximize their individual rewards in a collaborative (X.2.2) manner.…”

Attacking Deep Reinforcement Learning-Based Traffic Signal Control Systems with Colluding Vehicles

“…In terms of DRL, the concept of VANET has been introduced in multi-agent autonomous & connected vehicle setting. Recent work has demonstrated the effectiveness of dynamic graph information sharing mechanisms (Gunarathna et al 2019;Chen et al 2021;Wang et al 2020a).…”