Sensor-Based Navigation Using Hierarchical Reinforcement Learning

Gebauer, Christopher; Dengler, Nils; Bennewitz, Maren

doi:10.1007/978-3-031-22216-0_37

Cited by 4 publications

(2 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By decomposing tasks, HRL accelerates the learning process, as sub-tasks can be learned independently and in parallel, facilitating faster convergence to optimal policies [46]. This is particularly beneficial for real-time applications where rapid decision-making is crucial.…”

Section: Hierarchical Reinforcement Learning In the Aerial Robot's Ba...mentioning

confidence: 99%

Energy-Aware Hierarchical Reinforcement Learning Based on the Predictive Energy Consumption Algorithm for Search and Rescue Aerial Robots in Unknown Environments

Ramezani,

Amiri Atashgah

2024

Drones

View full text Add to dashboard Cite

Aerial robots (drones) offer critical advantages in missions where human participation is impeded due to hazardous conditions. Among these, search and rescue missions in disaster-stricken areas are particularly challenging due to the dynamic and unpredictable nature of the environment, often compounded by the lack of reliable environmental models and limited ground system communication. In such scenarios, autonomous aerial robots’ operation becomes essential. This paper introduces a novel hierarchical reinforcement learning-based algorithm to address the critical limitation of the aerial robot’s battery life. Central to our approach is the integration of a long short-term memory (LSTM) model, designed for precise battery consumption prediction. This model is incorporated into our HRL framework, empowering a high-level controller to set feasible and energy-efficient goals for a low-level controller. By optimizing battery usage, our algorithm enhances the aerial robot’s ability to deliver rescue packs to multiple survivors without the frequent need for recharging. Furthermore, we augment our HRL approach with hindsight experience replay at the low level to improve its sample efficiency.

show abstract

Section: Hierarchical Reinforcement Learning In the Aerial Robot's Ba...mentioning

confidence: 99%

Energy-Aware Hierarchical Reinforcement Learning Based on the Predictive Energy Consumption Algorithm for Search and Rescue Aerial Robots in Unknown Environments

Ramezani,

Amiri Atashgah

2024

Drones

View full text Add to dashboard Cite

show abstract

“…In cases featuring a hierarchy of hidden states, this problem can be classified as a hierarchical POMDP. Despite the success of hierarchical POMDP models as machine learning methods and their widespread use in artificial intelligence for facilitating adaptive behaviour [1][2][3][4][5][6] , our understanding of how the brain processes uncertain information to solve such challenges remains limited.…”

mentioning

confidence: 99%

Belief inference for hierarchical hidden states in spatial navigation

Katayama,

Shiraki,

Ishii

et al. 2024

Commun Biol

View full text Add to dashboard Cite

Uncertainty abounds in the real world, and in environments with multiple layers of unobservable hidden states, decision-making requires resolving uncertainties based on mutual inference. Focusing on a spatial navigation problem, we develop a Tiger maze task that involved simultaneously inferring the local hidden state and the global hidden state from probabilistically uncertain observation. We adopt a Bayesian computational approach by proposing a hierarchical inference model. Applying this to human task behaviour, alongside functional magnetic resonance brain imaging, allows us to separate the neural correlates associated with reinforcement and reassessment of belief in hidden states. The imaging results also suggest that different layers of uncertainty differentially involve the basal ganglia and dorsomedial prefrontal cortex, and that the regions responsible are organised along the rostral axis of these areas according to the type of inference and the level of abstraction of the hidden state, i.e. higher-order state inference involves more anterior parts.

show abstract

A Comprehensive Review of Mobile Robot Navigation Using Deep Reinforcement Learning Algorithms in Crowded Environments

Le,

Saeedvand,

Hsu

2024

J Intell Robot Syst

View full text Add to dashboard Cite

Navigation is a crucial challenge for mobile robots. Currently, deep reinforcement learning has attracted considerable attention and has witnessed substantial development owing to its robust performance and learning capabilities in real-world scenarios. Scientists leverage the advantages of deep neural networks, such as long short-term memory, recurrent neural networks, and convolutional neural networks, to integrate them into mobile robot navigation based on deep reinforcement learning. This integration aims to enhance the robot's motion control performance in both static and dynamic environments. This paper illustrates a comprehensive survey of deep reinforcement learning methods applied to mobile robot navigation systems in crowded environments, exploring various navigation frameworks based on deep reinforcement learning and their benefits over traditional simultaneous localization and mapping-based frameworks. Subsequently, we comprehensively compare and analyze the relationships and differences among three types of navigation: autonomous-based navigation, navigation based on simultaneous localization and mapping, and planning-based navigation. Moreover, the crowded environment includes static, dynamic, and a combination of obstacles in different typical application scenarios. Finally, we offer insights into the evolution of navigation based on deep reinforcement learning, addressing the problems and providing potential solutions associated with this emerging field.

show abstract

Sensor-Based Navigation Using Hierarchical Reinforcement Learning

Cited by 4 publications

References 9 publications

Energy-Aware Hierarchical Reinforcement Learning Based on the Predictive Energy Consumption Algorithm for Search and Rescue Aerial Robots in Unknown Environments

Energy-Aware Hierarchical Reinforcement Learning Based on the Predictive Energy Consumption Algorithm for Search and Rescue Aerial Robots in Unknown Environments

Belief inference for hierarchical hidden states in spatial navigation

A Comprehensive Review of Mobile Robot Navigation Using Deep Reinforcement Learning Algorithms in Crowded Environments

Contact Info

Product

Resources

About