Crowd-Aware Robot Navigation for Pedestrians with Multiple Collision Avoidance Strategies via Map-based Deep Reinforcement Learning

Abstract

It is challenging for a mobile robot to navigate through human crowds. Existing approaches usually assume that pedestrians follow a predefined collision avoidance strategy, like social force model (SFM) or optimal reciprocal collision avoidance (ORCA). However, their performances commonly need to be further improved for practical applications, where pedestrians follow multiple different collision avoidance strategies. In this paper, we propose a map-based deep reinforcement learning approach for crowd-aware robot navigation with various pedestrians. We use the sensor map to represent the environmental information around the robot, including its shape and observable appearances of obstacles. We also introduce the pedestrian map that specifies the movements of pedestrians around the robot. By applying both maps as inputs of the neural network, we show that a navigation policy can be trained to better interact with pedestrians following different collision avoidance strategies. We evaluate our approach under multiple scenarios both in the simulator and on an actual robot. The results show that our approach allows the robot to successfully interact with various pedestrians and outperforms compared methods in terms of the success rate.

RL-DOVS: Reinforcement Learning for Autonomous Robot Navigation in Dynamic Environments

“…Recent works on learning techniques to learn socially cooperative policies [ 10 , 11 , 12 ] incorporated pedestrian-collision-avoidance strategies without assuming any particular behavior rules, using a deep RL technique. In [ 13 ], the authors addressed the challenge of navigating through human crowds. They proposed a map-based deep RL approach using an environmental map that includes the robot shape and observable appearances of obstacles and a pedestrian map that specifies the movements of pedestrians around the robot.…”

“…Better training was obtained by applying both maps as inputs to the neural network. In [ 11 , 13 ], the system learns and tests the method in scenarios with a much lower number of moving obstacles than ours. In [ 10 ], the authors applied one specific scenario with a similar number of obstacles to ours.…”