Autonomous braking system via deep reinforcement learning

Abstract

In this paper, we propose a new autonomous braking system based on deep reinforcement learning. The proposed autonomous braking system automatically decides whether to apply the brake at each time step when confronting the risk of collision using the information on the obstacle obtained by the sensors. The problem of designing brake control is formulated as searching for the optimal policy in Markov decision process (MDP) model where the state is given by the relative position of the obstacle and the vehicle's speed, and the action space is defined as the set of the brake actions including 1) no braking, 2) weak, 3) mid, 4) strong brakiong actions. The policy used for brake control is learned through computer simulations using the deep reinforcement learning method called deep Q-network (DQN). In order to derive desirable braking policy, we propose the reward function which balances the damage imposed to the obstacle in case of accident and the reward achieved when the vehicle runs out of risk as soon as possible. DQN is trained for the scenario where a vehicle is encountered with a pedestrian crossing the urban road. Experiments show that the control agent exhibits desirable control behavior and avoids collision without any mistake in various uncertain environments.

A Survey of Deep Learning Applications to Autonomous Vehicle Control

“…One such reward function was proposed by Chae et al [111], who proposed an autonomous braking system for collision avoidance based on a DQN approach. The reward function balances two conflicting objectives: avoiding collision and getting out of high risk situations.…”

“…The test results demonstrated improved performance with the object-centric policy compared to models without attention or those based on heuristic object selection. Vision based techniques have also been used to mitigate collisions by Porav & Newman [129], who built on the previous work by Chae et al [111] by using a deep reinforcement learning algorithm for collision mitigation which can provide continuous control actions for both velocity and steering. The system uses a Variational AutoEncoder (VAE) coupled with an RNN to predict the movement of obstacles and learns a control policy with Deep Deterministic Policy Gradient (DDPG) to mitigate collisions in low TTC scenarios.…”

Multi-Objective Autonomous Braking System using Naturalistic Dataset

“…Chae et al propose an autonomous braking system using DRL [7]. In the study, the agent's sensors receive information about the pedestrian's position and adapts the brake control to the state change in a way that minimizes the chance of a collision.…”

“…where at time step t, v t is the velocity of the vehicle, dist t is the distance in meters between the vehicle and the pedestrian, and a t is the value of the brake action chosen and 1(x = y) has a value of 1 if the statement inside is true and 0 otherwise. The first term is included from the reward function proposed by Chae et al [7], describing the penalty given to the agent should in the case of an accident. The penalty is proportional to the velocity of the vehicle, reflecting the severity of the accident and therefore encouraging the vehicle to slow down in the case that an accident is unavoidable.…”

Safe Reinforcement Learning with Scene Decomposition for Navigating Complex Urban Environments

“…Reinforcement learning (RL) has been proposed as a way to automatically generate effective behaviors. RL has been applied to autonomous braking strategies at crosswalks [5], *This work was supported by the Honda Research Institute. 1 lane changing policies [6], and intersection navigation [7], [8].…”