RL-DOVS: Reinforcement Learning for Autonomous Robot Navigation in Dynamic Environments

Abstract

Autonomous navigation in dynamic environments where people move unpredictably is an essential task for service robots in real-world populated scenarios. Recent works in reinforcement learning (RL) have been applied to autonomous vehicle driving and to navigation around pedestrians. In this paper, we present a novel planner (reinforcement learning dynamic object velocity space, RL-DOVS) based on an RL technique for dynamic environments. The method explicitly considers the robot kinodynamic constraints for selecting the actions in every control period. The main contribution of our work is to use an environment model where the dynamism is represented in the robocentric velocity space as input to the learning system. The use of this dynamic information speeds the training process with respect to other techniques that learn directly either from raw sensors (vision, lidar) or from basic information about obstacle location and kinematics. We propose two approaches using RL and dynamic obstacle velocity (DOVS), RL-DOVS-A, which automatically learns the actions having the maximum utility, and RL-DOVS-D, in which the actions are selected by a human driver. Simulation results and evaluation are presented using different numbers of active agents and static and moving passive agents with random motion directions and velocities in many different scenarios. The performance of the technique is compared with other state-of-the-art techniques for solving navigation problems in environments such as ours.

Long-Range Navigation in Complex and Dynamic Environments with Full-Stack S-DOVS

“…We test and compare the use of the complete integrated system with the S-DOVS planner on its own in different maps, our proposed waypoint generator with others of the state of the art and extensively evaluate the system in both simulation and the real world. Note that we don't compare the S-DOVS planner with other local reactive planners such as RL-DOVS [22], as developing a local planner is not the goal of the paper, and any other local reactive planner could be used in the system instead of S-DOVS. Common reactive planners, by themselves, are limited to handling only a few meters of navigation within a single room in the real world.…”

“…In recent years, motion planning in dynamic environments has been tackled by learning approaches such as reinforcement learning, as demonstrated in various works, includ-ing [19][20][21], as well as by [22,23], which also incorporated the DOVS model. However, these methods face limitations due to the complexity of the real world and the large number of variables involved.…”