Dyna-: A heuristic planning reinforcement learning algorithm applied to role-playing game strategy decision systems

Santos, Matilde; Martı́n, Humberto; López, Victoria; Botella, Guillermo

doi:10.1016/j.knosys.2011.09.008

Cited by 47 publications

(18 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Models using simulated experience [c.f. Sutton and Barto (1998)] to improve valuations of explicitly represented system states exist in the form of reinforcement learning Dyna-based algorithms [e.g., Santos et al (2012) and Lowe and Ziemke (2013)]. However, these algorithms are limited to updating (either randomly or heuristically) already experienced states and do not simulate novel paths.…”

Section: Discussionmentioning

confidence: 99%

Finding Your Way from the Bed to the Kitchen: Reenacting and Recombining Sensorimotor Episodes Learned from Human Demonstration

et al. 2016

View full text Add to dashboard Cite

Several simulation theories have been proposed as an explanation for how humans and other agents internalize an "inner world" that allows them to simulate interactions with the external real world -prospectively and retrospectively. Such internal simulation of interaction with the environment has been argued to be a key mechanism behind mentalizing and planning. In the present work, we study internal simulations in a robot acting in a simulated human environment. A model of sensory-motor interactions with the environment is generated from human demonstrations and tested on a Robosoft Kompaï robot. The model is used as a controller for the robot, reproducing the demonstrated behavior. Information from several different demonstrations is mixed, allowing the robot to produce novel paths through the environment, toward a goal specified by top-down contextual information. The robot model is also used in a covert mode, where the execution of actions is inhibited and perceptions are generated by a forward model. As a result, the robot generates an internal simulation of the sensory-motor interactions with the environment. Similar to the overt mode, the model is able to reproduce the demonstrated behavior as internal simulations. When experiences from several demonstrations are combined with a top-down goal signal, the system produces internal simulations of novel paths through the environment. These results can be understood as the robot imagining an "inner world" generated from previous experience, allowing it to try out different possible futures without executing actions overtly. We found that the success rate in terms of reaching the specified goal was higher during internal simulation, compared to overt action. These results are linked to a reduction in prediction errors generated during covert action. Despite the fact that the model is quite successful in terms of generating covert behavior toward specified goals, internal simulations display different temporal distributions compared to their overt counterparts. Links to human cognition and specifically mental imagery are discussed.

show abstract

Section: Discussionmentioning

confidence: 99%

Finding Your Way from the Bed to the Kitchen: Reenacting and Recombining Sensorimotor Episodes Learned from Human Demonstration

et al. 2016

View full text Add to dashboard Cite

show abstract

“…In 'LunarLanderContinuous-v2', state vector consist of 8 features which we used only 4 of them in cloud control and fuzzy control systems, these are: the horizontal and vertical coordinates s 1 and s 2 , the vertical velocity s 4 and the angle s 5 . Action is two real values vector.…”

Section: Lunarlandercontinuous-v2mentioning

confidence: 99%

“…Bianchi et al [4] also proved the convergence of heuristic exploration and the boundary of error estimation in theoretical analysis. Besides, Santo et al [5] proposed Dyna-H algorithm based on Dyna-Q framework, using the A* algorithm as a heuristic function, providing guidance for the planning part of the algorithm. Miyazaki [6] proposed the k-certainty exploration method which builds a maximum likelihood model of the state transition probabilities to guide action selection.…”

Section: Introductionmentioning

confidence: 99%

A Study of Qualitative Knowledge-Based Exploration for Continuous Deep Reinforcement Learning

Cao

Liu

et al. 2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYAs an important method to solve sequential decisionmaking problems, reinforcement learning learns the policy of tasks through the interaction with environment. But it has difficulties scaling to largescale problems. One of the reasons is the exploration and exploitation dilemma which may lead to inefficient learning. We present an approach that addresses this shortcoming by introducing qualitative knowledge into reinforcement learning using cloud control systems to represent 'if-then' rules. We use it as the heuristics exploration strategy to guide the action selection in deep reinforcement learning. Empirical evaluation results show that our approach can make significant improvement in the learning process.

show abstract

“…Data Tamer [27] in MIT, which deals even with data consolidation, is a good example. And not only tools for monitoring, integrate, store or analyze data but also for organizing e-learning [12,25]. All of these tools were working over a server or a huge system that need a very important first investment.…”

Section: Big Data Systems In Health and Social Sciencesmentioning

confidence: 99%

Big+Open Data: Some applications for a Smartcity

López

Miñana

Sánchez

et al. 2015

2015 IEEE International Conference on Progress in Informatics and Computing (PIC)

View full text Add to dashboard Cite

The way of thinking on Big Data, Open Data and their use by organizations or individuals has been a trending topic over the last few years. Big Data deals with collecting, storing, analyzing and putting data in value. Big, medium and small enterprises want to include information technologies in their management and decision processes. At the same time, movements about rights on public data have increased their presence and force. Data from governments must be open. Every day, more and more cities and countries are opening their data. Open Data has emerged as a new paradigm for the public service provision model with a special role in Smart City. The main goal of Big and Open Data in a Smart City is to develop systems which can be useful for citizens. In this work we analyze how both private enterprises and governments manage to improve their data value by combining private and public datasets and we give some examples of our work in this area.

show abstract

Dyna-: A heuristic planning reinforcement learning algorithm applied to role-playing game strategy decision systems

Cited by 47 publications

References 25 publications

Finding Your Way from the Bed to the Kitchen: Reenacting and Recombining Sensorimotor Episodes Learned from Human Demonstration

Finding Your Way from the Bed to the Kitchen: Reenacting and Recombining Sensorimotor Episodes Learned from Human Demonstration

A Study of Qualitative Knowledge-Based Exploration for Continuous Deep Reinforcement Learning

Big+Open Data: Some applications for a Smartcity

Contact Info

Product

Resources

About