Real-Time Object Navigation With Deep Neural Networks and Hierarchical Reinforcement Learning

Staroverov, Aleksey; Yudin, Dmitry; Belkin, Ilya; Adeshkin, Vasily; Solomentsev, Yaroslav; Panov, Aleksandr I.

doi:10.1109/access.2020.3034524

Cited by 38 publications

(10 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…HRL could be formalized as global and local policies. In sparse rewards settings, a methods with two levels of policies was demonstrated by this works (Chaplot et al 2020b) (Chaplot et al 2020a) (Staroverov et al 2020).…”

Section: Related Workmentioning

confidence: 93%

Landmark Policy Optimization for Object Navigation Task

Staroverov,

Panov

2021

Preprint

Self Cite

View full text Add to dashboard Cite

This work studies object goal navigation task, which involves navigating to the closest object related to the given semantic category in unseen environments. Recent works have shown significant achievements both in the end-to-end Reinforcement Learning approach and modular systems, but need a big step forward to be robust and optimal. We propose a hierarchical method that incorporates standard task formulation and additional area knowledge as landmarks, with a way to extract these landmarks. In a hierarchy, a low level consists of separately trained algorithms to the most intuitive skills, and a high level decides which skill is needed at this moment. With all proposed solutions, we achieve a 0.75 success rate in a realistic Habitat simulator. After a small stage of additional model training in a reconstructed virtual area at a simulator, we successfully confirmed our results in a real-world case.

show abstract

Section: Related Workmentioning

confidence: 93%

Landmark Policy Optimization for Object Navigation Task

Staroverov,

Panov

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Hsu et al [54] divided the complex indoor environment into different local areas, and generated navigation actions based on the scene image and target location. The latest research interests also include hierarchical RL [55] and the graph structure neural network [56] . Indoor navigation is becoming increasingly practical.…”

Section: Developmentmentioning

confidence: 99%

Deep reinforcement learning based mobile robot navigation: A review

Zhu

Zhang

2021

Tsinghua Sci. Technol.

256

View full text Add to dashboard Cite

Navigation is a fundamental problem of mobile robots, for which Deep Reinforcement Learning (DRL) has received significant attention because of its strong representation and experience learning abilities. There is a growing trend of applying DRL to mobile robot navigation. In this paper, we review DRL methods and DRL-based navigation frameworks. Then we systematically compare and analyze the relationship and differences between four typical application scenarios: local obstacle avoidance, indoor navigation, multi-robot navigation, and social navigation. Next, we describe the development of DRL-based navigation. Last, we discuss the challenges and some possible solutions regarding DRL-based navigation.

show abstract

“…However, retraining is required after changing targets as it has been designed for specific tasks. Staroverov et al [25] proposed hierarchical DRL with classical methods integrated into the model. The navigation model presented by [26] is an endto-end neural network trained with a combination of expert demonstrations, imitation learning, and reinforcement learning.…”

Section: Learning-based Navigationmentioning

confidence: 99%

AdaptiveON: Adaptive Outdoor Local Navigation Method For Stable and Reliable Actions

Ji¹,

Weerakoon²,

Guan³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present a novel outdoor navigation algorithm to generate stable and efficient actions to navigate a robot to the goal. We use a multi-stage training pipeline and show that our model produces policies that result in stable and reliable robot navigation on complex terrains. Based on the Proximal Policy Optimization (PPO) algorithm, we developed a novel method to achieve multiple capabilities for outdoor navigation tasks, namely: alleviating the robot's drifting, keeping the robot stable on bumpy terrains, avoiding climbing on hills with steep elevation changes, and collision avoidance. Our training process mitigates the reality(sim-to-real) gap by introducing more generalized environmental and robotic parameters and training with rich features of Lidar perception in the Unity simulator. We evaluate our method in both simulation and the real world with Clearpath Husky and Jackal. Additionally, we compare our method against the state-of-the-art approaches and show that in the real world it improves stability by at least 30.7% on uneven terrains, reduces drifting by 8.08%, and for high hills our trained policy keeps small changes of the elevation of the robot at each motion step by preventing the robot from moving on areas with high gradients.

show abstract

Real-Time Object Navigation With Deep Neural Networks and Hierarchical Reinforcement Learning

Cited by 38 publications

References 42 publications

Landmark Policy Optimization for Object Navigation Task

Landmark Policy Optimization for Object Navigation Task

Deep reinforcement learning based mobile robot navigation: A review

AdaptiveON: Adaptive Outdoor Local Navigation Method For Stable and Reliable Actions

Contact Info

Product

Resources

About