k-Certainty Exploration Method: an action selector to identify the environment in reinforcement learning

Miyazaki, Kazuteru; Yamamura, Masatoshi; Kobayashi, Shinya

doi:10.1016/s0004-3702(96)00062-8

Cited by 28 publications

(8 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RL is the study of programs that improve their performance and adapt an agent to an unknown environment by receiving rewards and punishments [19].…”

Section: Reinforcement Learningmentioning

confidence: 99%

Applying reinforcement learning method to optimize an Energy Hub operation in the smart grid

Rayati

Sheikhi

Ranjbar

2015

2015 IEEE Power &Amp; Energy Society Innovative Smart Grid Technologies Conference (ISGT)

View full text Add to dashboard Cite

New days, the concepts of "Smart Grid" and "Energy Hub" have been introduced to improve the operation of the energy systems. This paper introduces a new conception entitling Smart Energy Hub (S. E. Hub), as a multi-carrier energy system in a smart grid environment. To show the application of this novel idea, we present a residential S. E. Hub which employs Reinforcement Learning (RL) method for finding a near optimal solution. The simulation results show that by applying the S. E. Hub model and then using the proposed method for a residential customer, running cost is reduced substantially. While, comparing with the classical ones, the RL method does not require any data about the environment and either equipment's parameters. Index Terms--Smart Energy Hub (S. E. Hub), Smart Grids, Reinforcement Learning (RL), Energy Management System, OptimizationI.

show abstract

“…RL is the study of programs that improve their performance and adapt an agent to an unknown environment by receiving rewards and punishments [19].…”

Section: Reinforcement Learningmentioning

confidence: 99%

Applying reinforcement learning method to optimize an Energy Hub operation in the smart grid

Rayati

Sheikhi

Ranjbar

2015

2015 IEEE Power &Amp; Energy Society Innovative Smart Grid Technologies Conference (ISGT)

View full text Add to dashboard Cite

show abstract

“…Longer training is necessary for RL to converge when the demand is stochastic. The k-Certainty Exploration method was used to track the use of the learned rules and assign a k-certainty factor to the tracked rules [13]. Using this approach, uncertainty over the learned rules is reduced and the environment can be identified gradually.…”

Section: Related Workmentioning

confidence: 99%

Fast Reinforcement Learning under Uncertainties with Self-Organizing Neural Networks

Teng

Tan

2015

2015 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT)

View full text Add to dashboard Cite

Using feedback signals from the environment, a reinforcement learning (RL) system typically discovers action policies that recommend actions effective to the states based on a Q-value function. However, uncertainties over the estimation of the Q-values can delay the convergence of RL. For fast RL convergence by accounting for such uncertainties, this paper proposes several enhancements to the estimation and learning of the Q-value using a self-organizing neural network. Specifically, a temporal difference method known as Q-learning is complemented by a Q-value Polarization procedure, which contrasts the Q-values using feedback signals on the effect of the recommended actions. The polarized Q-values are then learned by the self-organizing neural network using a Bi-directional Template Learning procedure. Furthermore, the polarized Qvalues are in turn used to adapt the reward vigilance of the ART-based self-organizing neural network using a Bi-directional Adaptation procedure. The efficacy of the resultant system called Fast Learning (FL) FALCON is illustrated using two singletask problem domains with large MDPs. The experiment results from these problem domains unanimously show FL-FALCON converging faster than the compared approaches.

show abstract

“…3 We did not give KITTY a learning mechanism. Therefore KITTY's action selection probability was stable.…”

Section: Application To Othellomentioning

confidence: 99%

Development of a reinforcement learning system to play Othello

Miyazaki

Tsuboi²,

Kobayashi

2004

Artificial Life and Robotics

Self Cite

View full text Add to dashboard Cite

The purpose of the reinforcement learning system is to learn an optimal policy in general. On the other hand, in two-player games such as Othello, it is important to acquire a penalty-avoiding policy that can avoid losing the game. We know the penalty avoiding rational policy making algorithm (PARP) to learn the policy. If we apply PARP to large-scale problems, we are confronted with an explosion of the number of states. In this article, we focus on Othello, a game that has huge state spaces. We introduce several ideas and heuristics to adapt PARP to Othello. We show that our learning player beats the well-known Othello program, KITTY.

show abstract

k-Certainty Exploration Method: an action selector to identify the environment in reinforcement learning

Cited by 28 publications

References 16 publications

Applying reinforcement learning method to optimize an Energy Hub operation in the smart grid

Applying reinforcement learning method to optimize an Energy Hub operation in the smart grid

Fast Reinforcement Learning under Uncertainties with Self-Organizing Neural Networks

Development of a reinforcement learning system to play Othello

Contact Info

Product

Resources

About