A Sample Aggregation Approach to Experiences Replay of Dyna-Q Learning

Shi, Haobin; Yang, Shike; Hwang, Kao-Shing; Chen, Jialin; Hu, Mengkai; Zhang, Hengsheng

doi:10.1109/access.2018.2847048

Cited by 8 publications

(3 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, solely relying on interactions with the real world is sometimes inefficient. Inspired by the Dyna structure [28], the historical experience may be able to provide more guidance for learning by generating virtual experience. The Dyna structure is further explained in Appendix A.…”

Section: M-carla Algorithmmentioning

confidence: 99%

A Reinforcement-learning-based Bidding Strategy for Power Suppliers with Limited Information

Jia

Yan

et al. 2022

Journal of Modern Power Systems and Clean Energy

View full text Add to dashboard Cite

The power market is a typical imperfectly competitive market where power suppliers gain higher profits through strategic bidding behaviors. Most existing studies assume that a power supplier is accessible to the sufficient market information to derive an optimal bidding strategy. However, this assumption may not be true in reality, particularly when a power market is newly launched. To help power suppliers bid with the limited information, a modified continuous action reinforcement learning automata algorithm is proposed. This algorithm introduces the discretization and Dyna structure into continuous action reinforcement learning automata algorithm for easy implementation in a repeated game. Simulation results verify the effectiveness of the proposed learning algorithm.

show abstract

Section: M-carla Algorithmmentioning

confidence: 99%

A Reinforcement-learning-based Bidding Strategy for Power Suppliers with Limited Information

Jia

Yan

et al. 2022

Journal of Modern Power Systems and Clean Energy

View full text Add to dashboard Cite

show abstract

“…In order to control the randomness of action selection, a simulated annealing (SA) algorithm [23,24] is used to optimize softmax function. The softmax function is a method for balancing Exploration and exploitation [25] in the RL method, which chooses the action according to the average reward of each action, and the probability of the action a t being chosen is higher if the average reward produced by the action is higher than the average reward produced by the other action.…”

Section: Softmax Function Based On Simulated Annealingmentioning

confidence: 99%

Fuzzy Reinforcement Learning and Curriculum Transfer Learning for Micromanagement in Multi-Robot Confrontation

Meng

2019

Information

View full text Add to dashboard Cite

Multi-Robot Confrontation on physics-based simulators is a complex and time-consuming task, but simulators are required to evaluate the performance of the advanced algorithms. Recently, a few advanced algorithms have been able to produce considerably complex levels in the context of the robot confrontation system when the agents are facing multiple opponents. Meanwhile, the current confrontation decision-making system suffers from difficulties in optimization and generalization. In this paper, a fuzzy reinforcement learning (RL) and the curriculum transfer learning are applied to the micromanagement for robot confrontation system. Firstly, an improved Qlearning in the semi-Markov decision-making process is designed to train the agent and an efficient RL model is defined to avoid the curse of dimensionality. Secondly, a multi-agent RL algorithm with parameter sharing is proposed to train the agents. We use a neural network with adaptive momentum acceleration as a function approximator to estimate the state-action function. Then, a method of fuzzy logic is used to regulate the learning rate of RL. Thirdly, a curriculum transfer learning method is used to extend the RL model to more difficult scenarios, which ensures the generalization of the decision-making system. The experimental results show that the proposed method is effective.

show abstract

“…The crowd would be out of control effortlessly when density exceeds a certain threshold, whereas public security would be threatened seriously. The evacuation path is scheduled to condense the evacuation time in thickly populated areas, especially complex environments with obstacles and multiple path is one of the significant issues of crowd simulations caused by emergency tragedies [1]. In such a condition, most pedestrians will track to the nearer exit or follow the crowd to reach exit [2], the above will cause overcrowding delay, trampling mortalities and other safety incidents.…”

Section: Introductionmentioning

confidence: 99%

Improved Particle Swam Optimization for Crowd Simulation Using Hybrid Agent Reinforcement Learning Algorithm

Pavithra¹,

Radhamani²,

Thangavel³

2021

ITII

View full text Add to dashboard Cite

In an emergency route planning technique, simulating the dynamic crowd has route capacity constraints and global target of evacuating all crowd evacuees. To stimulate the crowd, the new arena is developed to know the real-time situation to face the crowd evacuation on exit point. The crowd evacuation is done with the process of Hybrid Agent Reinforcement Learning (HARL) algorithm consisting of Improved Multi-Agent Reinforcement Learning (IMARL) and State-Action-Reward-State-Action (SARSA). In the proposed work, the appropriate route selection mechanism focused on finding optimum evacuation route(s) is done in the first phase. Dynamic crowd can also be evacuated to find its way with the support of the HARL process in the second phase. The proposed HARL method can also be implemented with multi-objective improved particle swarm optimization (IPSO) technique for crowd simulation. The experimented results demonstrate the effectiveness of stability in the HARL process, which provides an improved performance for crowd simulation

show abstract

A Sample Aggregation Approach to Experiences Replay of Dyna-Q Learning

Cited by 8 publications

References 29 publications

A Reinforcement-learning-based Bidding Strategy for Power Suppliers with Limited Information

A Reinforcement-learning-based Bidding Strategy for Power Suppliers with Limited Information

Fuzzy Reinforcement Learning and Curriculum Transfer Learning for Micromanagement in Multi-Robot Confrontation

Improved Particle Swam Optimization for Crowd Simulation Using Hybrid Agent Reinforcement Learning Algorithm

Contact Info

Product

Resources

About