Deep Q-Learning Network with Bayesian-Based Supervised Expert Learning

Kim, Chayoung

doi:10.3390/sym14102134

Cited by 4 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To optimize the training network, the difference between the evaluated network and the target network is calculated as the following loss function [32]:…”

Section: E-dqn Training Of Dronesmentioning

confidence: 99%

E-DQN-Based Path Planning Method for Drones in Airsim Simulator under Unknown Environment

Chao,

Dillmann,

Roennau

et al. 2024

Biomimetics

View full text Add to dashboard Cite

To improve the rapidity of path planning for drones in unknown environments, a new bio-inspired path planning method using E-DQN (event-based deep Q-network), referring to introducing event stream to reinforcement learning network, is proposed. Firstly, event data are collected through an airsim simulator for environmental perception, and an auto-encoder is presented to extract data features and generate event weights. Then, event weights are input into DQN (deep Q-network) to choose the action of the next step. Finally, simulation and verification experiments are conducted in a virtual obstacle environment built with an unreal engine and airsim. The experiment results show that the proposed algorithm is adaptable for drones to find the goal in unknown environments and can improve the rapidity of path planning compared with that of commonly used methods.

show abstract

“…To optimize the training network, the difference between the evaluated network and the target network is calculated as the following loss function [32]:…”

Section: E-dqn Training Of Dronesmentioning

confidence: 99%

E-DQN-Based Path Planning Method for Drones in Airsim Simulator under Unknown Environment

Chao,

Dillmann,

Roennau

et al. 2024

Biomimetics

View full text Add to dashboard Cite

show abstract

“…The method set a threshold based on the maximum LR value, LR min and LR max as values smaller than the threshold, and as the median of the minimum values of all recorded measures of the relative change in the MI between epochs. In this study, we do not suggest a separate "LR Range Test", and if prior knowledge can be obtained by the study [43], the system designer defines LR min and LR max . The study [18] suggested that the components were known to fit the data a priori; therefore, we might remove them based on prior experience.…”

Section: Proposed Algorithmmentioning

confidence: 99%

Target-Network Update Linked with Learning Rate Decay Based on Mutual Information and Reward in Deep Reinforcement Learning

Kim

2023

Symmetry

Self Cite

View full text Add to dashboard Cite

In this study, a target-network update of deep reinforcement learning (DRL) based on mutual information (MI) and rewards is proposed. In DRL, updating the target network from the Q network was used to reduce training diversity and contribute to the stability of learning. If it is not properly updated, the overall update rate is reduced to mitigate this problem. Simply slowing down is not recommended because it reduces the speed of the decaying learning rate. Some studies have been conducted to improve the issues with the t-soft update based on the Student’s-t distribution or a method that does not use the target-network. However, there are certain situations in which using the Student’s-t distribution might fail or force it to use more hyperparameters. A few studies have used MI in deep neural networks to improve the decaying learning rate and directly update the target-network by replaying experiences. Therefore, in this study, the MI and reward provided in the experience replay of DRL are combined to improve both the decaying learning rate and the target-network updating. Utilizing rewards is appropriate for use in environments with intrinsic symmetry. It has been confirmed in various OpenAI gymnasiums that stable learning is possible while maintaining an improvement in the decaying learning rate.

show abstract

Multi-Agent Chronological Planning with Model-Agnostic Meta Reinforcement Learning

Zhu

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

In this study, we propose an innovative approach to address a chronological planning problem involving the multiple agents required to complete tasks under precedence constraints. We model this problem as a stochastic game and solve it with multi-agent reinforcement learning algorithms. However, these algorithms necessitate relearning from scratch when confronted with changes in the chronological order of tasks, resulting in distinct stochastic games and consuming a substantial amount of time. To overcome this challenge, we present a novel framework that incorporates meta-learning into a multi-agent reinforcement learning algorithm. This approach enables the extraction of meta-parameters from past experiences, facilitating rapid adaptation to new tasks with altered chronological orders and circumventing the time-intensive nature of reinforcement learning. Then, the proposed framework is demonstrated through the implementation of a method named Reptile-MADDPG. The performance of the pre-trained model is evaluated using average rewards before and after fine-tuning. Our method, in two testing tasks, improves the average rewards from −44 to −37 through 10,000 steps of fine-tuning in two testing tasks, significantly surpassing the two baseline methods that only attained −51 and −44, respectively. The experimental results demonstrate the superior generalization capabilities of our method across various tasks, thus constituting a significant contribution towards the design of intelligent unmanned systems.

show abstract

Deep Q-Learning Network with Bayesian-Based Supervised Expert Learning

Cited by 4 publications

References 15 publications

E-DQN-Based Path Planning Method for Drones in Airsim Simulator under Unknown Environment

E-DQN-Based Path Planning Method for Drones in Airsim Simulator under Unknown Environment

Target-Network Update Linked with Learning Rate Decay Based on Mutual Information and Reward in Deep Reinforcement Learning

Multi-Agent Chronological Planning with Model-Agnostic Meta Reinforcement Learning

Contact Info

Product

Resources

About