Merging in Congested Freeway Traffic Using Multipolicy Decision Making and Passive Actor-Critic Learning

Abstract

Freeway merging in congested traffic is a significant challenge toward fully automated driving. Merging vehicles need to decide not only how to merge into a spot, but also where to merge. We present a method for the freeway merging based on multi-policy decision making with a reinforcement learning method called passive actorcritic (pAC), which learns with less knowledge of the system and without active exploration. The method selects a merging spot candidate by using the state value learned with pAC. We evaluate our method using real traffic data. Our experiments show that pAC achieves 92% success rate to merge into a freeway, which is comparable to human decision making.

Kernel-Based Hamilton–Jacobi Equations for Data-Driven Optimal and H-Infinity Control

“…Compared to existing methods, the proposed method offers several advantages, as described in Section VII. In future work, the proposed method should be extended for applications involving the control of unknown nonlinear systems, e.g., autonomous vehicles involving human interactions [1]. The realization of effective control requires the development of mathematical models of such unknown systems, regarding which only limited information is available.…”

“…Let us focus on a partially unknown nonlinear system for which the input matrix B is known but a true drift term f tr (x) is unknown. For example, such a partially unknown system can arise owing to the interaction between autonomous vehicles and manually operated vehicles [1]. A partially unknown system can be expressed as…”

“…There exist various unknown nonlinear systems that it is useful to control. Examples of such systems are (semi-)autonomous vehicles that comprise a human driver with unknown nonlinear dynamics [1], [2] and batteries in electric vehicles that should be managed by taking into account their unknown dynamics [3]. It is desirable that such systems are controlled optimally and safely, albeit their dynamics remain partially unknown.…”

Speed harmonisation and merge control using connected automated vehicles on a highway lane closure: a reinforcement learning approach

“…Several control approaches have been developed for the speed harmonisation using Q‐learning (QL) [13], model predictive control (MPC) [14], and proportional–integral feedback [16]. The MC alleviates the congestion at the merging area by assigning priority to vehicles [17–19]. Some methods have been proposed for the MC using a heuristic algorithm [18] or a reinforcement learning (RL) [19].…”

“…The MC alleviates the congestion at the merging area by assigning priority to vehicles [17–19]. Some methods have been proposed for the MC using a heuristic algorithm [18] or a reinforcement learning (RL) [19]. Distinguishing from the above studies, we consider the mixed traffic environment and combine the mechanisms of speed harmonisation and MC to improve the traffic flow and fuel consumption.…”