Reinforcement Learning-Based Traffic Control: Mitigating the Adverse Impacts of Control Transitions

Abstract

An important aspect of automated driving is to handle situations where it fails or is not allowed in specific traffic situations. This case study explores means, by which control transitions in a mixed autonomy system can be organized in order to minimize their adverse impact on traffic flow. We assess a number of different approaches for a coordinated management of transitions, covering classic traffic management paradigms and AI-driven controls. We demonstrate that they yield excellent results when compared to a do-nothing scenario. This text further details a model for control transitions that is the basis for the simulation study presented. The results encourage the deployment of reinforcement learning on the control problem for a scenario with mandatory take-over requests.Index Terms-connected automated vehicles (CAV), reinforcement learning (RL), take-over request (ToR), traffic management (TM), transition of control (ToC) NOMENCLATURE AV Automated vehicle CAV Connected automated vehicle CV Connected vehicle LoD Level of demand MDP Markov decision process MRM Minimum risk manoeuvre MV Manual vehicle No-AD No automated driving RL Reinforcement learning RSI Roadside infrastructure TM Traffic management TMC Traffic management center ToC Transition of control ToR Take-over request

Control Transitions in Level 3 Automation: Safety Implications in Mixed-Autonomy Traffic

“…At least two come to mind: (i) the vehicle may issue a ToR when still within the ODD so that the ToC can take place just at the end of the ODD (if this is known in advance by the vehicle), or (ii) traffic management in combination with vehicle-to-infrastructure communication (V2I) can provide an optimal schedule for AVs to performs their ToCs at specific times and positions. Our previous study [34] considered some of those aspects, especially with the objective to maintain automated driving as long as possible. Clearly, such management considerations could also apply in terms of traffic safety, but these were not specifically addressed in this work.…”

“…This analysis focused on the severity of changes in these driving scenarios and derived potential benefits from automated driving functions by projecting the results to a national scale. In a simulation-based case study aimed to propose potential traffic management countermeasures to mitigate adverse impacts induced by control transitions [34], our earlier research focused on traffic efficiency performance, deferring an in-depth analysis of safety ramifications from ToCs. Therefore in Section 3, we introduce a new case study dedicated to investigating these safety effects in greater detail, and we share our findings in Section 4.…”

A Systematic Survey of Control Techniques and Applications in Connected and Automated Vehicles

“…Autonomous vehicles (AVs) and connected and automated vehicles (CAVs) as the key instantiation of automated driving and collaborative driving automation become more and more widely deployed, and they benefit both individuals and society [22]- [27]. In the past few decades, manufacturers and researchers have been thriving in automotive industrialization to maximize benefits from automated driving and collaborative driving automation [28]- [34]. The hierarchical framework, including perception, planning, and control modules, is the popular pipeline for both automated driving and collaborative driving automation techniques [35].…”