Double action Q-learning for obstacle avoidance in a dynamically changing environment

Abstract

In this paper, we propose a new method for sobing the reinforcement learning problem in a dynamically changing environment, as inivehicle navigatiop, in which the Markov Decision Process used in traditional reinforcement learning is modified so that the response of the environment is taken into consideratiin for determining the agent's next state. This i s achieved by changing the action-value function to handle three parameters at a time, namely, the current state, action taken by the agent, and action taken by the environment. As it considers the actions by the agent and environment, it is termed "Double Action". Based on the Q-learning method, the proposed method is implemented and the update rule is modified to handle all of the three parameters. Preliminary results show that the proposed method has the sum of rewards (negative) 89.5% less than that of the traditional method. Apart form that, our new method also has the total number of collisions and mean steps used in one episode 89.5% and 15.5% lower than that of the traditional method respectively.

Pedestrian Behavior Prediction based on Motion Patterns for Vehicle-to-Pedestrian Collision Avoidance

“…1. In this paper, we assume that (a) the proposed method runs in a vehicle, which is to perform CA and other tasks in a DCE; (b) the method for determining the vehicle's action decision is outside the scope of this paper (Interested reader should refer to [12] for details); (c) an observable pedestrian instance is a distance value measured at time t by the vehicle's sensors; and (d) pedestrian behavior is defined by a series of pedestrian motions. To start with, when a group of pedestrian instances at time t are observed, they are first associated with existing trajectories that have been assembled through the previous t-1 time steps.…”