Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion

Ha, Paul Young Joun; Chen, Sikai; Dong, Jiqian; Du, Runjia; Li, Yujie; Labi, Samuel

doi:10.48550/arxiv.2010.05436

Cited by 16 publications

(32 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, multi-agent reinforcement learning (MARL) has been greatly advanced and successfully applied to a variety of complex multi-agent systems such as games [10], traffic light control [11] and fleet management [12]. The applications of MARL to autonomous driving also exist [13][14][15][16], with the objective of accomplishing autonomous driving tasks cooperatively and reacting timely to HDVs. In particular, previous works [15,17] also apply the MARL to highway lane change tasks and show promising and scalable performance, in which AVs learn cooperatively via sharing the same objective (i.e., reward/cost function) that considers safety and efficiency.…”

Section: Introductionmentioning

confidence: 99%

“…The applications of MARL to autonomous driving also exist [13][14][15][16], with the objective of accomplishing autonomous driving tasks cooperatively and reacting timely to HDVs. In particular, previous works [15,17] also apply the MARL to highway lane change tasks and show promising and scalable performance, in which AVs learn cooperatively via sharing the same objective (i.e., reward/cost function) that considers safety and efficiency. However, those reward designs often ignore the passengers' comfort, which may lead to sudden acceleration and deceleration that can cause ride discomfort.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic

Zhou,

Chen,

Yan

et al. 2021

Preprint

View full text Add to dashboard Cite

Autonomous driving has attracted significant research interests in the past two decades as it offers many potential benefits, including releasing drivers from exhausting driving and mitigating traffic congestion, among others. Despite promising progress, lane-changing remains a great challenge for autonomous vehicles (AV), especially in mixed and dynamic traffic scenarios. Recently, reinforcement learning (RL), a powerful datadriven control method, has been widely explored for lane-changing decision makings in AVs with encouraging results demonstrated. However, the majority of those studies are focused on a single-vehicle setting, and lane-changing in the context of multiple AVs coexisting with humandriven vehicles (HDVs) have received scarce attention. In this paper, we formulate the lane-changing decision making of multiple AVs in a mixed-traffic highway environment as a multi-agent reinforcement learning (MARL) problem, where each AV makes lane-changing decisions based on the motions of both neighboring AVs and HDVs. Specifically, a multi-agent advantage actor-critic network (MA2C) is developed with

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic

Zhou,

Chen,

Yan

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In this study, we focus on the second type as the target traffic jam, and consider a bottleneck on a highway road. Researchers have treated various bottleneck types or effects: fixed bottleneck (such as traffic accidents, work zones, and obstacles) [38,39,[51][52][53], lane reduction [49,54,55], on-ramp [45][46][47]53], rubbernecking in a specific region [25], imposition of capacity drop [27,56,57], and sag [58][59][60][61]. Among various bottlenecks on highways, this study focuses on sag.…”

Section: Introductionmentioning

confidence: 99%

“…Various strategies manipulate special vehicles to improve highway traffic at bottlenecks. Examples of them include trajectory planning [49], changing the settings of adaptive cruise control according to the traffic situation [46], running slowly upstream of traffic jam [38,39,47,53,58,59,61], maneuvers combining deceleration and acceleration [60], maneuvers produced by reinforcement learning [54,55], reinforcement learning of merge control and speed harmonization [52], combination of cooperative adaptive cruise control platooning, cooperative merge, and speed harmonization [45], and the maneuvers aiming to remove the traffic jam such as Pacer Cars [22], speed harminization [57], dynamic VSL with CVs [56], and JAD [25,27]. Among these strategies, this study focuses on JAD.…”

Section: Introductionmentioning

confidence: 99%

System-size dependence of a jam-absorption driving strategy to remove traffic jam caused by a sag under the presence of traffic instability

Nishi,

Watanabe

2021

Preprint

View full text Add to dashboard Cite

Sag is a road section where a downhill changes into an uphill, and is a highway bottleneck. This study elucidates the influence of the system size (the number of vehicles in a system) on a jam-absorption driving (JAD) strategy to remove the traffic jam caused by a sag, in a system where all vehicles are connected vehicles (CVs) and run on a single-lane road with a sag. Our strategy detects the downstream front of the traffic jam (DFTJ) using CVs as floating sensors, and assigns the nearest CV upstream of the DFTJ by a preset distance or more to start the JAD maneuver: running toward the goal near the future disappearing time-space point of the DFTJ. Every time step, we detect the DFTJ, and update the goal and this CV's maneuver. After removing the jam by passing the goal position, this CV resumes its car-following behavior, and we assign another CV to start the JAD maneuver to remove the next jam. Numerical simulations by a car-following model possessing the traffic instability have evaluated the Maximum reduction in the Average total Travel time per vehicle (MAT), and that in the Average total Fuel consumption per vehicle (MAF) by varying the preset distance for a given system size ranging from 500 to 10000 vehicles. Numerical results have revealed that MAT and MAF increase with respect to the system size. The rate of increase in MAT increases slightly with respect to the system size, while that in MAF decreases with respect to the system size till roughly 5000 vehicles, and is roughly stable with fluctuation for larger system sizes. The optimal preset distances for MAT and MAF become roughly constant as the system size increases, and the former is much shorter than the latter. Therefore, to realize MAT and MAF simultaneously is not possible. Nevertheless, realizing MAF leads to a considerable reduction in the average total travel time per vehicle for large system sizes.

show abstract

“…(LE-CPSs) [31,34] for perception, prediction, planning, control, etc. In particular, neural networks may greatly improve performance and efficiency for planning and general decision making in LE-CPSs, such as autonomous driving [11], human robot interaction [10], smart grid [20] and smart buildings [35]. Moreover, compared with traditional model-based approaches, they can save the time and effort of explicitly modeling systems with complex dynamics and significant uncertainties.…”

mentioning

confidence: 99%

Physics-Aware Safety-Assured Design of Hierarchical Neural Network based Planner

Huang¹,

Wang²,

Zheng³

et al. 2022

Preprint

View full text Add to dashboard Cite

Neural networks have shown great promises in planning, control, and general decision making for learning-enabled cyber-physical systems (LE-CPSs), especially in improving performance under complex scenarios. However, it is very challenging to formally analyze the behavior of neural network based planners for ensuring system safety, which significantly impedes their applications in safety-critical domains such as autonomous driving. In this work, we propose a hierarchical neural network based planner that analyzes the underlying physical scenarios of the system and learns a system-level behavior planning scheme with multiple scenariospecific motion-planning strategies. We then develop an efficient verification method that incorporates overapproximation of the system state reachable set and novel partition and union techniques for formally ensuring system safety under our physics-aware planner. With theoretical analysis, we show that considering the different physical scenarios and building a hierarchical planner based on such analysis may improve system safety and verifiability. We also empirically demonstrate the effectiveness of our approach and its advantage over other baselines in practical case studies of unprotected left turn and highway merging, two common challenging safety-critical tasks in autonomous driving. CCS CONCEPTS• Computer systems organization → Robotic autonomy; • Software and its engineering → Formal software verification.

show abstract

Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion

Cited by 16 publications

References 0 publications

Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic

Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic

System-size dependence of a jam-absorption driving strategy to remove traffic jam caused by a sag under the presence of traffic instability

Physics-Aware Safety-Assured Design of Hierarchical Neural Network based Planner

Contact Info

Product

Resources

About