Learning General Planning Policies from Small Examples Without Supervision

Abstract

Generalized planning is concerned with the computation of general policies that solve multiple instances of a planning domain all at once. It has been recently shown that these policies can be computed in two steps: first, a suitable abstraction in the form of a qualitative numerical planning problem (QNP) is learned from sample plans, then the general policies are obtained from the learned QNP using a planner. In this work, we introduce an alternative approach for computing more expressive general policies which does not require sample plans or a QNP planner. The new formulation is very simple and can be cast in terms that are more standard in machine learning: a large but finite pool of features is defined from the predicates in the planning examples using a general grammar, and a small subset of features is sought for separating “good” from “bad” state transitions, and goals from non-goals. The problems of finding such a “separating surface” while labeling the transitions as “good” or “bad” are jointly addressed as a single combinatorial optimization problem expressed as a Weighted Max-SAT problem. The advantage of looking for the simplest policy in the given feature space that solves the given examples, possibly non-optimally, is that many domains have no general, compact policies that are optimal. The approach yields general policies for a number of benchmark domains.

Learning General Optimal Policies with Graph Neural Networks: Expressive Power, Transparency, and Limits

“…The second that when the gripper is not empty, any action that makes H false and does not affect n should be selected. It has been shown that general policies of this form can be learned without supervision by solving a Max-Weighted SAT theory T (S, F ) where S is a set of sampled state transitions, and F is a large but finite pool of Boolean and numerical features obtained from the domain predicates (Francès, Bonet, and Geffner 2021).…”

“…General Policies. The problem of learning general policies has been addressed using combinatorial approaches where the symbolic domains are given (Khardon 1999;Martín and Geffner 2004;Bonet, Francès, and Geffner 2019;Francès, Bonet, and Geffner 2021), DL approaches where the domains are given too (Toyer et al 2020;Garg, Bajpai, and Mausam 2020), and DRL approaches that do not make use of prior knowledge about the structure of either domains or states (Groshev et al 2018;Chevalier-Boisvert et al 2019;Campero et al 2021). This work is a step to bring the first two approaches together along with their potential benefits.…”

“…Even in simple tasks, such as retrieving a key to open a door in a simple environment, they may require a large number of simulations, and even then, they may fail to generalize to all possible situations (Chevalier-Boisvert et al 2019). Interestingly, the computation of general policies has been addressed recently in a model-based setting that assumes that a general model of the actions is known in terms of action schemas and predicates (Bonet and Geffner 2018;Francès, Bonet, and Geffner 2021). This paper is a step aimed at bringing these threads together with two motivations: to replace the combinatorial methods that have been proposed to learn general policies by more robust and scalable deep learning methods, and to do so in a principled manner where the intermediate representations and experimental results, both positive and negative, can be understood.…”

Learning Sketches for Decomposing Planning Problems into Subproblems of Bounded Width

“…We turn to the problem of learning sketches given a set of instances P of the target class of problems Q and the desired bound k on sketch width. We roughly follow the approach for learning general policies (Bonet, Francès, and Geffner 2019;Francès, Bonet, and Geffner 2021) by constructing a theory T k,m (P, F ) from P, k, a bound m on the number of sketch rules, and a finite pool of features F obtained from the domain predicates and a fixed grammar.…”

“…The language of sketches is powerful, as sketches can encode everything from simple goal serializations to full general policies. Indeed, the language of general policies is the language of sketches but with a slightly different semantics where the subgoal states s to be reached from a state s are restricted to be one step away from s (Bonet and Geffner 2018;Francès, Bonet, and Geffner 2021). More interestingly, sketches can split problems into subproblems of bounded width (Lipovetzky and Geffner 2012;Lipovetzky 2021) which can then be solved greedily, in polynomial time, by a variant of the SIW algorithm, called SIW R (Bonet and Geffner 2021).…”