A Selective Macro-learning Algorithm and its Application to the NxN Sliding-Tile Puzzle

Finkelstein, Lev; Markovitch, Shaul

doi:10.1613/jair.484

Cited by 14 publications

(9 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our random-walk bootstrapping is most similar to the approach used in Micro-Hillary (Finkelstein & Markovitch, 1998), a macro-learning system for problem solving. In that work, instead of generating problems via random walks starting at an initial state, random walks were generated backward from goal states.…”

Section: Related Workmentioning

confidence: 99%

Approximate Policy Iteration with a Policy Language Bias: Solving Relational Markov Decision Processes

Fern¹,

Yoon²,

Givan³

2006

jair

109

View full text Add to dashboard Cite

We study an approach to policy selection for large relational Markov Decision Processes (MDPs). We consider a variant of approximate policy iteration (API) that replaces the usual value-function learning step with a learning step in policy space. This is advantageous in domains where good policies are easier to represent and learn than the corresponding value functions, which is often the case for the relational MDPs we are interested in. In order to apply API to such problems, we introduce a relational policy language and corresponding learner. In addition, we introduce a new bootstrapping routine for goalbased planning domains, based on random walks. Such bootstrapping is necessary for many large relational MDPs, where reward is extremely sparse, as API is ineffective in such domains when initialized with an uninformed policy. Our experiments show that the resulting system is able to find good policies for a number of classical planning domains and their stochastic variants by solving them as extremely large relational MDPs. The experiments also point to some limitations of our approach, suggesting future work.

show abstract

Section: Related Workmentioning

confidence: 99%

Approximate Policy Iteration with a Policy Language Bias: Solving Relational Markov Decision Processes

Fern¹,

Yoon²,

Givan³

2006

jair

109

View full text Add to dashboard Cite

show abstract

“…This can be generalized to the N × N version of Eight Puzzle and the corresponding stage structure. Similar to the illustration by Finkelstein and Markovitch (1998), the city-block distance heuristic coupled with the IDA * algorithm limits the search by first bringing the tile in question to a nearest location to its goal position without disturbing previously arranged tiles, and then doing a bounded search to place that tile and any previously arranged tiles in their goal positions.…”

Section: Resultsmentioning

confidence: 99%

“…Our algorithm does not need the correct ordering of features in the macro‐table to be given, but instead uses the exercises to infer this ordering. More recently Finkelstein and Markovitch (1998) implemented a macro‐learning algorithm for an N × N generalization of Eight Puzzle. They also exploit the idea of learning from easier problems first, thus validating the approach taken here.…”

Section: Discussion and Related Workmentioning

confidence: 99%

Learning to Solve Problems From Exercises

Tadepalli

2008

Computational Intelligence

View full text Add to dashboard Cite

It is a common observation that learning easier skills makes it possible to learn the more difficult skills. This fact is routinely exploited by parents, teachers, textbook writers, and coaches. From driving, to music, to science, there hardly exists a complex skill that is not learned by gradations. Natarajan's model of "learning from exercises" captures this kind of learning of efficient problem solving skills using practice problems or exercises (Natarajan 1989). The exercises are intermediate subproblems that occur in solving the main problems and span all levels of difficulty. The learner iteratively bootstraps what is learned from simpler exercises to generalize techniques for solving more complex exercises. In this paper, we extend Natarajan's framework to the problem reduction setting where problems are solved by reducing them to simpler problems. We theoretically characterize the conditions under which efficient learning from exercises is possible. We demonstrate the generality of our framework with successful implementations in the Eight Puzzle, symbolic integration, and simulated robot planning domains illustrating three different representations of control knowledge, namely, macro-operators, control rules, and decision lists. The results show that the learning rates for the exercises framework are competitive with those for learning from problems solved by the teacher.

show abstract

“…The preconditions and effects of the extended se-1 Our usage of this term is the same as in the literature on learning macro-rules [11], but in this paper, macro-rules are used only for analysis, not as new move options that are available at run time. quence, p * and a * , are constructed as follows.…”

Section: Rule Compositionmentioning

confidence: 99%

Automatic move pruning for single-agent search

Holte

Burch

2014

AI Communications

View full text Add to dashboard Cite

Move pruning is a low-overhead technique for reducing search cost in single-agent search problems by avoiding the generation of duplicate states. Redundant sequences of moves, where the effect of one sequence is provably identical to some other sequence of moves, are suppressed during search. We present an algorithm for automatically identifying redundant move sequences in a general class of single-agent search problems, and a method for selecting redundant move sequences to prune during search. We demonstrate that the redundant move sequences which are to be pruned must be chosen carefully, and give experimental results using our move pruning method which show a speedup of multiple orders of magnitude in a variety of domains. Finally, we give theoretical results on conditions where move pruning does, and does not, affect the correctness of different search algorithms.

show abstract

A Selective Macro-learning Algorithm and its Application to the NxN Sliding-Tile Puzzle

Cited by 14 publications

References 32 publications

Approximate Policy Iteration with a Policy Language Bias: Solving Relational Markov Decision Processes

Approximate Policy Iteration with a Policy Language Bias: Solving Relational Markov Decision Processes

Learning to Solve Problems From Exercises

Automatic move pruning for single-agent search

Contact Info

Product

Resources

About