Visual Navigation in Real-World Indoor Environments Using End-to-End Deep Reinforcement Learning

Abstract

Visual navigation is essential for many applications in robotics, from manipulation, through mobile robotics to automated driving. Deep reinforcement learning (DRL) provides an elegant map-free approach integrating image processing, localization, and planning in one module, which can be trained and therefore optimized for a given environment. However, to date, DRL-based visual navigation was validated exclusively in simulation, where the simulator provides information that is not available in the real world, e.g., the robot's position or segmentation masks. This precludes the use of the learned policy on a real robot. Therefore, we present a novel approach that enables a direct deployment of the trained policy on real robots. We have designed a new powerful simulator capable of domain randomization. To facilitate the training, we propose visual auxiliary tasks and a tailored reward scheme. The policy is fine-tuned on images collected from real-world environments. We have evaluated the method on a mobile robot in a real office environment. The training took approximately 30 hours on a single GPU. In 30 navigation experiments, the robot reached a 0.3-meter neighbourhood of the goal in more than 86.7 % of cases. This result makes the proposed method directly applicable to tasks like mobile manipulation.Index Terms-Vision-based navigation, reinforcement learning, deep learning methods. I. INTRODUCTIONV ISION-BASED navigation is essential for a broad range of robotic applications, from industrial and service robotics to automated driving. The wide-spread use of this technique will be further stimulated by the availability of lowcost cameras and high-performance computing hardware.Conventional vision-based navigation methods usually build a map of the environment and then use planning to reach

Reinforcement and Curriculum Learning for Off-Road Navigation of an UGV with a 3D LiDAR

“…In [ 28 ], an unmanned aerial vehicle is trained with Gazebo to fly among obstacles with a 2D LiDAR. For indoor navigation of UGVs, methods can be found where the main exteroceptive sensors are a 2D rangefinders [ 29 , 30 , 31 ], depth cameras with a limited field of view [ 32 ] or RGB cameras [ 9 , 33 ]. In [ 34 ], 2D virtual range data generated from a monocular camera is employed by a UGV as input for RL.…”

“…Finally, it is relevant to mention that most of the previously cited papers about RL adopt an Actor–Critic scheme [ 9 , 27 , 28 , 29 , 31 , 33 , 35 , 36 , 37 , 38 ].…”

“…RL has been applied to a wide range of areas, such as games [ 4 ], computer vision [ 5 ] or robotics [ 6 ]. In the latter, RL has been successfully employed for manipulator control [ 7 ], for indoor navigation [ 8 , 9 ] and for self-driving cars [ 10 , 11 ], but little research has been made on off-road scenarios.…”

Indoor Target-Driven Visual Navigation based on Spatial Semantic Information

“…The target-driven navigation approach is a new DRL-based end-to-end navigation approach, first proposed by Zhu et al [1] in 2017. This approach takes only the image of the current scene and the target object as input, and generates an action in the 3D environment as output [10]- [16]. Hence, when an agent takes its next action, the condition is current state and the target, not just its current state, so there is no need to retrain the model for the new target.…”

Stronger Generalization Guarantees for Robot Learning by Combining Generative Models and Real-World Data

“…Adversarial techniques have also been developed for generating challenging environments [21], [22]. Prior work has also explored augmenting realworld training with large amounts of procedurally generated environments via domain adaptation techniques [23], transfer learning [24], or fine-tuning [25]. We highlight that none of the methods above provide guarantees on generalization to real-world environments.…”