Safe Lane-Changing in CAVs Using External Safety Supervisors: A Review

Abstract

Connected autonomous vehicles (CAVs) can exploit information received from other vehicles in addition to their sensor information to make decisions. For this reason, their deployment is expected to improve traffic safety and efficiency. Safe lane-changing is a significant challenge for CAVs, particularly in mixed traffic, i.e. with human-driven vehicles (HDVs) on the road, as the set of vehicles around them varies very quickly, and they can only communicate with a fraction of them. Many approaches have been proposed, with most recent work adopting a multi-agent reinforcement learning (MARL) approach, but those do not provide safety guarantees making them unsuitable for such a safety-critical application. A number of external safety techniques for reinforcement learning have been proposed, such as shielding, control barrier functions, model predictive control and recovery RL, but those have not been applied to CAV lane changing.This paper investigates whether external safety supervisors could be used to provide safety guarantees for MARL-based CAV lane changing (LC-CAV). For this purpose, a MARL approach to CAV lane changing (MARL-CAV) is designed, using parameter sharing and a replay buffer to motivate cooperative behaviour and collaboration among CAVs. This is then used as a baseline to discuss the applicability of the state-of-the-art external safety techniques for reinforcement learning to MARL-CAV. Comprehensive analysis shows that integrating an external safety technique to MARL for lane changing in CAVs is challenging, and none of the existing external safety techniques can be directly applied to MARL-CAV as these safety techniques require prior knowledge of unsafe states and recovery policies.

Multi-agent reinforcement learning for safe lane changes by connected and autonomous vehicles: A survey

“…Recently, a set of requirements for a safe CAV controller was proposed by Lenka et al for high-level lane change decision making. These requirements include support for multi-agent interactions, mixed traffic scenario, discrete action space, prior knowledge of unsafe states, and backup policies [18]. Similarly, the low-level controller of CAVs has some significant requirements in terms of environment, control parameters, deployment, and safety.…”

“…The evaluation of this method shows promising results with zero collisions, but it only includes a simplified environment. Moreover, this method considers interaction with only two agents, and would therefore need to be significantly extended to be applied to CAV controllers in mixed traffic [18].…”

“…Although the safety methods presented can ensure level II safety and above, they require accurate formalisation of the safety constraints. In complex applications such as CAV driving, it could be challenging to formulate safety constraints [18]. Therefore, a critical analysis of the compatibility of these safety methods with CAV motion controllers is necessary to integrate them.…”

“…In this safety method, constraints are formulated for a simple simulation of two agents. Therefore, extending them to the CAV domain can be a challenging task [18] and would require in-depth analysis to formulate a CBF function. The second safety method is a stability supervisor which regulates partial derivatives of a policy to ensure safety [68].…”

“…To evaluate the applicability of safety methods to CAV lane changing, Lenka et. al defines a set of requirements for safety methods [18], but, these requirements are not intended to evaluate state-of-the-art MARL CAV controllers.…”