Learning and Reusing Goal-Specific Policies for Goal-Driven Autonomy

Jaidee, Ulit; Muñoz-Ávila, Héctor; Aha, David W.

doi:10.1007/978-3-642-32986-9_15

Cited by 15 publications

(14 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The central concept of GDA is that goals should have an option to be updated, or reformulated, once plans go awry, based on some reasoning. While (Muñoz-Avila et al ., 2010) follow a rule-based goal formulation scheme, LGDA (Jaidee et al ., 2011a; Jaidee et al ., 2011b) learns (discrepancy, goal) pairs by experience. (Jaidee et al ., 2012) Is perhaps the most advanced GDA formulation where the agent also learns new alternative goals and goal-based policies from experience.…”

Section: Relevant Workmentioning

confidence: 99%

Towards life-long adaptive agents: using metareasoning for combining knowledge-based planning with situated learning

Parashar

Goel

Sheneman³

et al. 2018

The Knowledge Engineering Review

View full text Add to dashboard Cite

We consider task planning for long-living intelligent agents situated in dynamic environments. Specifically, we address the problem of incomplete knowledge of the world due to the addition of new objects with unknown action models. We propose a multilayered agent architecture that uses meta-reasoning to control hierarchical task planning and situated learning, monitor expectations generated by a plan against world observations, forms goals and rewards for the situated reinforcement learner, and learns the missing planning knowledge relevant to the new objects. We use occupancy grids as a low-level representation for the high-level expectations to capture changes in the physical world due to the additional objects, and provide a similarity method for detecting discrepancies between the expectations and the observations at run-time; the meta-reasoner uses these discrepancies to formulate goals and rewards for the learner, and the learned policies are added to the hierarchical task network plan library for future re-use. We describe our experiments in the Minecraft and Gazebo microworlds to demonstrate the efficacy of the architecture and the technique for learning. We test our approach against an ablated reinforcement learning (RL) version, and our results indicate this form of expectation enhances the learning curve for RL while being more generic than propositional representations.

show abstract

Section: Relevant Workmentioning

confidence: 99%

Towards life-long adaptive agents: using metareasoning for combining knowledge-based planning with situated learning

Parashar

Goel

Sheneman³

et al. 2018

The Knowledge Engineering Review

View full text Add to dashboard Cite

show abstract

“…GDA-C has some characteristics in common with GRL (Jaidee et al, 2012), which also uses RL for goal formulation. However, GRL is a single agent system and, unlike GDA-C, cannot scale to play complete RTS games.…”

Section: Related Workmentioning

confidence: 99%

“…To test this claim we conducted an empirical evaluation using the Wargus RTS environment in which we compared the performance of GDA-C versus CLASS QL (Jaidee et al, 2012), an ablation of GDA-C where the RL agents coordinate by sharing only the same reward function. We first compared GDA-C and CLASS QL indirectly by testing both against the built-in AI in Wargus, a proficient AI that comes with the game and is designed to be competitive versus a mid-range player.…”

Section: Introductionmentioning

confidence: 99%

“…EISBot plays complete games by using six managers (e.g., for building an economy, combat), only one of which uses GDA (i.e., it selects which units to produce). The GDA system GRL (Jaidee et al, 2012) plays RTS game scenarios were each side starts with a fixed number of units. No buildings are allowed and hence no new units can be produced, which drastically reduces the GRL's state and action space.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Case-Based Goal-Driven Coordination of Multiple Learning Agents

Jaidee

Muñoz-Ávila

Aha

2013

Case-Based Reasoning Research and Development

Self Cite

View full text Add to dashboard Cite

Abstract.Although several recent studies have been published on goal reasoning (i.e., the study of agents that can self-select their goals), none have focused on the task of learning and acting on large state and action spaces. We introduce GDA-C, a case-based goal reasoning algorithm that divides the state and action space among cooperating learning agents. Cooperation between agents emerges because (1) they share a common reward function and (2) GDA-C formulates the goal that each agent needs to achieve. We claim that its case-based approach for goal formulation is critical to the agents' performance. To test this claim we conducted an empirical study using the Wargus RTS environment, where we found that GDA-C outperforms its non-GDA ablation.

show abstract

“…Frequently, the expectation X is defined as the expected state. In this situation a discrepancy occurs if X  s' holds (e.g., (Jaidee et al, 2012)). In our work a discrepancy happens when the attempt to accomplish a goal fails.…”

Section: Definitionsmentioning

confidence: 99%

Case-Based Learning of Applicability Conditions for Stochastic Explanations

Finestrali

Muñoz-Ávila

2013

Case-Based Reasoning Research and Development

Self Cite

View full text Add to dashboard Cite

Abstract. This paper studies the problem of explaining events in stochastic environments. We explore three ideas to address this problem: (1) Using the notion of Stochastic Explanation, which associates with any event a probability distribution over possible plausible explanations for the event. (2) Retaining as cases (event, stochastic explanation) pairs when unprecedented events occur. (3) Learning the probability distribution in the stochastic explanation as cases are reused. We claim that a system using stochastic explanations reacts faster to abrupt changes in the environment than a system using deterministic explanations. We demonstrate this claim in a CBR system, incorporating the 3 ideas above, while playing a real-time strategy game. We observe how the CBR system when using stochastic explanations reacts faster to abrupt changes in the environment than when using deterministic explanations.

show abstract

Learning and Reusing Goal-Specific Policies for Goal-Driven Autonomy

Cited by 15 publications

References 11 publications

Towards life-long adaptive agents: using metareasoning for combining knowledge-based planning with situated learning

Towards life-long adaptive agents: using metareasoning for combining knowledge-based planning with situated learning

Case-Based Goal-Driven Coordination of Multiple Learning Agents

Case-Based Learning of Applicability Conditions for Stochastic Explanations

Contact Info

Product

Resources

About