Relevance Grounding for Planning in Relational Domains

Lang, Tobias; Toussaint, Marc

doi:10.1007/978-3-642-04180-8_65

Cited by 18 publications

(12 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Notable examples include reasoning only in terms of the subset of objects that are relevant for current planning purposes (relevance grounding) [12], or using variables to stand in for the objects relevant to the current actions (deictic references) [13].…”

Section: Related Workmentioning

confidence: 99%

What good are actions? Accelerating learning using learned action priors

Rosman

Ramamoorthy²

2012

2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL)

View full text Add to dashboard Cite

The computational complexity of learning in sequential decision problems grows exponentially with the number of actions available to the agent at each state. We present a method for accelerating this process by learning action priors that express the usefulness of each action in each state. These are learned from a set of different optimal policies from many tasks in the same state space, and are used to bias exploration away from less useful actions. This is shown to improve performance for tasks in the same domain but with different goals. We extend our method to base action priors on perceptual cues rather than absolute states, allowing the transfer of these priors between tasks with differing state spaces and transition functions, and demonstrate experimentally the advantages of learning with action priors in a reinforcement learning context.

show abstract

Section: Related Workmentioning

confidence: 99%

What good are actions? Accelerating learning using learned action priors

Rosman

Ramamoorthy²

2012

2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL)

View full text Add to dashboard Cite

show abstract

“…Indeed, feature learning in general is an open and difficult question, which has been considered in many contexts, e.g., [11] and [12]. The possible features include the state label (as discussed previously), as well as any sensory information the agent may receive from the environment.…”

Section: B Feature Selectionmentioning

confidence: 99%

“…Examples include reasoning only in terms of the subset of objects that are relevant for current planning purposes (relevance grounding) [12], or using variables to stand in for the objects relevant to the current actions (deictic references) [28]. This is similar to the way in which pruning is used in search, but we prune based on the expected utility of the action, estimated from its utility in the optimal policies for a set of previous tasks.…”

Section: Related Workmentioning

confidence: 99%

Action Priors for Learning Domain Invariances

Rosman

Ramamoorthy

2015

IEEE Trans. Auton. Mental Dev.

View full text Add to dashboard Cite

An agent tasked with solving a number of different decision making problems in similar environments has an opportunity to learn over a longer timescale than each individual task. Through examining solutions to different tasks, it can uncover behavioral invariances in the domain, by identifying actions to be prioritized in local contexts, invariant to task details. This information has the effect of greatly increasing the speed of solving new problems. We formalise this notion as action priors, defined as distributions over the action space, conditioned on environment state, and show how these can be learnt from a set of value functions. We apply action priors in the setting of reinforcement learning, to bias action selection during exploration. Aggressive use of action priors performs context based pruning of the available actions, thus reducing the complexity of lookahead during search. We additionally define action priors over observation features, rather than states, which provides further flexibility and generalizability, with the additional benefit of enabling feature selection. Action priors are demonstrated in experiments in a simulated factory environment and a large random graph domain, and show significant speed ups in learning new tasks. Furthermore, we argue that this mechanism is cognitively plausible, and is compatible with findings from cognitive psychology.

show abstract

“…In contrast to the current paper, however, they assume a given model of the world. Recently, Lang and Toussaint [17] and Joshi et al [12] have shown that successful planning typically involves only a small subset of relevant objects respectively states and how to make use of this fact to speed up symbolic dynamic programming significantly. A principled approach to exploration, however, has not been developed.…”

Section: Related Workmentioning

confidence: 99%

Exploration in Relational Worlds

Lang

Toussaint

Kersting

2010

Machine Learning and Knowledge Discovery in Databases

Self Cite

View full text Add to dashboard Cite

One of the key problems in model-based reinforcement learning is balancing exploration and exploitation. Another is learning and acting in large relational domains, in which there is a varying number of objects and relations between them. We provide one of the first solutions to exploring large relational Markov decision processes by developing relational extensions of the concepts of the Explicit Explore or Exploit (E 3) algorithm. A key insight is that the inherent generalization of learnt knowledge in the relational representation has profound implications also on the exploration strategy: what in a propositional setting would be considered a novel situation and worth exploration may in the relational setting be an instance of a well-known context in which exploitation is promising. Our experimental evaluation shows the effectiveness and benefit of relational exploration over several propositional benchmark approaches on noisy 3D simulated robot manipulation problems.

show abstract

Relevance Grounding for Planning in Relational Domains

Cited by 18 publications

References 13 publications

What good are actions? Accelerating learning using learned action priors

What good are actions? Accelerating learning using learned action priors

Action Priors for Learning Domain Invariances

Exploration in Relational Worlds

Contact Info

Product

Resources

About