Modeling adaptive platoon and reservation‐based intersection control for connected and autonomous vehicles employing deep reinforcement learning

Li, Duowei; Wu, Jianping; Zhu, Feng; Chen, Tianyi; Wong, Yiik Diew

doi:10.1111/mice.12956

Cited by 18 publications

(12 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a fundamental variable in the platoon-based control, the platoon size will affect the stability of platoon and the efficiency of passage in AIM, which is systematically analysed in [4]. And Li et al adopted Deep Q learning to determine the optimal platoon size in AIM [5]. Apart from the platoon formation among the adjacent vehicles in the same lanes, virtual platoon maps two-dimensional traffic flows in AIM to one-dimensional queue according to their distance to the center of the intersection, and groups non-conflicting vehicles to cross the intersection simultaneously [6,7].…”

Section: Literature Reviewmentioning

confidence: 99%

D-HAL: Distributed Hierarchical Adversarial Learning for Multi-Agent Interaction in Autonomous Intersection Management

Li¹,

Wu²,

He³

2023

Preprint

View full text Add to dashboard Cite

Autonomous Intersection Management (AIM) provides a signal-free intersection scheduling paradigm for Connected Autonomous Vehicles (CAVs). Distributed learning method has emerged as an attractive branch of AIM research. Compared with centralized AIM, distributed AIM can be deployed to CAVs at a lower cost, and compared with rule-based and optimization-based method, learningbased method can treat various complicated real-time intersection scenarios more flexibly. Deep reinforcement learning (DRL) is the mainstream approach in distributed learning to address AIM problems. However, the large-scale simultaneous interactive decision of multiple agents and the rapid changes of environment caused by interactions pose challenges for DRL, making its reward curve oscillating and hard to converge, and ultimately leading to a compromise in safety and computing efficiency. For this, we propose a non-RL learning framework, called Distributed Hierarchical Adversarial Learning (D-HAL). The framework includes an actor network that generates the actions of each CAV at each step. The immediate discriminator evaluates the interaction performance of the actor network at the current step, while the final discriminator makes the final evaluation of the overall trajectory from a series of interactions. In this framework, the long-term outcome of the behavior no longer motivates the actor network in terms of discounted rewards, but rather through a designed adversarial loss function with discriminative labels. The proposed model is evaluated at a four-way-six-lane intersection, and outperforms several state-of-the-art methods on ensuring safety and reducing travel time.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

D-HAL: Distributed Hierarchical Adversarial Learning for Multi-Agent Interaction in Autonomous Intersection Management

Li¹,

Wu²,

He³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Based on the high-quality and fine-grained path data provided by connected and autonomous vehicles (CAV), and Shi, Nie, et al (2022) proposed deep reinforcement learning methods to control CAV and buses, respectively, in a distributed manner. Likewise, Li et al (2023) develop a deep reinforcement learning-based model to control CAV platoon at signal-free intersections.…”

Section: Introductionmentioning

confidence: 99%

“…Likewise, Li et al. (2023) develop a deep reinforcement learning‐based model to control CAV platoon at signal‐free intersections.…”

Section: Introductionmentioning

confidence: 99%

Virtual‐real‐fusion simulation framework for evaluating and optimizing small‐spatial‐scale placement of cooperative roadside sensing units

Ma,

Zheng,

Wang

et al. 2024

Computer aided Civil Eng

View full text Add to dashboard Cite

Roadside sensing units’ (RSUs) perception capability may be substantially impaired by occlusion issue even they work cooperatively. However, the joint influence of static and dynamic occlusions in real‐life situations remains inadequately considered in optimizing RSUs’ placement. This study proposes a virtual‐real‐fusion simulation (VRFS) framework that combines traffic simulation and point clouds of real‐world road environment to optimize RSUs’ deployment. Point clouds and triangular meshes are used to model static and dynamic obstacles, respectively. A structure‐retained spherical projection method is developed to efficiently emulate RSUs’ data collection. Based on the developed VRFS, the probabilistic occupancy maps (POM) are created to represent traffic scenarios. The POM‐based cross entropy (CE) is proposed as the surrogate metric for evaluating the detection performance of cooperative RSUs. The Bayesian optimizer is applied to optimize the RSUs’ placement parameters (decision variables) by minimizing CE. Test results show that it is viable to use the POM‐based CE as a proxy for evaluating cooperative RSUs’ sensing performance. Considering the occlusion effect adds to the efficacy of POM‐based CE as a surrogate metric. Compared with traffic volume, the adverse effect of the proportion of large vehicles on RSUs’ detection performance is more significant. There are no significant patterns regarding how the optimized RSU positions vary with traffic parameters. The comparisons with existing methods further verify the importance of considering both static and dynamic occlusions in optimizing RSUs’ placement. Besides, the proposed method can yield better optimization results more efficiently than existing approaches.

show abstract

“…CAVs not only provide high‐resolution, multidimensional data for traffic control (Massaro et al., 2017; Z. Zhang et al., 2022) but also render it possible to jointly optimize traffic signals and vehicle trajectories (Q. Guo et al., 2019). Numerous control methods have emerged, with a common goal of enhancing signal operations through the consideration of approaching vehicle trajectories (Feng et al., 2015; W. Li & Ban, 2019; D. Li et al., 2023; Liang et al., 2020; W. Ma et al., 2020; Wang et al., 2021) or planning vehicle trajectories based on signal status (He et al., 2015; S. E. Li et al., 2015; Stebbins et al., 2017; X. Wu et al., 2015; Zhou et al., 2017). Furthermore, the other studies focus on the joint optimization of traffic signals and vehicle trajectories, known as signal‐vehicle coupled control (SVCC; Feng et al., 2018; Y. Guo et al., 2019; Z. Li et al., 2014; Liu et al., 2022; C. Ma et al., 2023; Soleimaniamiri et al., 2020; Yu et al., 2018).…”

Section: Introductionmentioning

confidence: 99%

A spatiotemporal control method at isolated intersections under mixed‐autonomy traffic conditions

Dai,

Ding,

et al. 2023

Computer aided Civil Eng

View full text Add to dashboard Cite

With the introduction of connected and automated vehicles (CAVs), the integrated control of traffic signals, lane assignments, and vehicle trajectories becomes feasible, offering notable benefits for enhancing intersection operations. However, during the prolonged transition to an entirely CAV environment, how to fully leverage the advantage of CAVs while considering the characteristics of human‐driven vehicles remains a huge challenge. To address this challenge, this paper proposes a joint optimization method for spatiotemporal resources at isolated intersections under mixed‐autonomy traffic conditions. Initially, the lane assignment optimization problem is modeled as a mixed integer linear program model to maximize the reserve capacity. Subsequently, the signal‐vehicle coupled control is formulated as a dynamic programming model with the objective of reducing vehicle travel time. Additionally, criteria are established to assess the need for re‐optimizing lane assignments. Simulations validate the superiority of the proposed control method over adaptive control in terms of traffic efficiency and intersection capacity amid substantial traffic demand fluctuations. Sensitivity analyses reveal that the proposed control method can yield higher benefits under medium traffic demand levels. Furthermore, the proposed algorithm exhibits no significant sensitivity to the CAV market adoption rate, suggesting its applicability throughout the CAV adoption process.

show abstract

Modeling adaptive platoon and reservation‐based intersection control for connected and autonomous vehicles employing deep reinforcement learning

Cited by 18 publications

References 56 publications

D-HAL: Distributed Hierarchical Adversarial Learning for Multi-Agent Interaction in Autonomous Intersection Management

D-HAL: Distributed Hierarchical Adversarial Learning for Multi-Agent Interaction in Autonomous Intersection Management

Virtual‐real‐fusion simulation framework for evaluating and optimizing small‐spatial‐scale placement of cooperative roadside sensing units

A spatiotemporal control method at isolated intersections under mixed‐autonomy traffic conditions

Contact Info

Product

Resources

About