Neural Network Heuristic Functions for Classical Planning: Bootstrapping and Comparison to Other Methods

Abstract

How can we train neural network (NN) heuristic functions for classical planning, using only states as the NN input? Prior work addressed this question by (a) per-instance imitation learning and/or (b) per-domain learning. The former limits the approach to instances small enough for training data generation, the latter to domains where the necessary knowledge generalizes across instances. Here we explore three methods for (a) that make training data generation scalable through bootstrapping and approximate value iteration. In particular, we introduce a new bootstrapping variant that estimates search effort instead of goal distance, which as we show converges to the perfect heuristic under idealized circumstances. We empirically compare these methods to (a) and (b), aligning three different NN heuristic function learning architectures for cross-comparison in an experiment of unprecedented breadth in this context. Key lessons are that our methods and imitation learning are highly complementary; that per-instance learning often yields stronger heuristics than per-domain learning; and the LAMA planner is still dominant but our methods outperform it in one benchmark domain.

Sampling from Pre-Images to Learn Heuristic Functions for Classical Planning (Extended Abstract)

“…The coverage of a planner is defined as the percent of initial states for which a solution path is found within the given planning budget. Ferber et al (2021) report observing that in general the coverage superiority between the different NN heuristics tested did not vary over time. That is, the planning time used and the relative coverage superiority between the algorithms were not correlated.…”

“…The baseline option for performing the rollout, and the method used by Ferber et al (2021), is to randomly select actions a for which applying the regression operator is valid. In addition to testing RSL using random action selection we instantiate a version of RSL we name Novelty guided Regression based Learning (N-RSL) that aims to increase the structural diversity of operators selected in its regression.…”

“…Our benchmark set of domains, instances and initial states is the "Hard Task" set as introduced by Ferber et Ferber et al (2021). According to standard single thread CPU benchmarks, our vCPU can be 20% faster.…”

“…In this work, we introduce the Regression based Supervised Learning (RSL) algorithm in order to learn per instance NN defined heuristic functions. Like other methods (Ferber et al 2021;Yu, Kuroiwa, and Fukunaga 2020) do, RSL selects a set of regressions, trajectories of sets of states, or preimages, found via the application of well-known and efficient pre-imaging operators (Rintanen 2008) that rely on symbolic action descriptions. These trajectories found along a given regression, always start from the set of goal states of an instance, and then training states are sampled from each set along the trajectory.…”

Neural Network Heuristic Functions: Taking Confidence into Account

“…Successes include the AlphaGo series (Silver et al 2016(Silver et al , 2017(Silver et al , 2018, as well as heuristic search for single-agent games such as Rubik's Cube (Agostinelli et al 2019). Given the prominence of heuristic search in AI Planning (Hoffmann and Nebel 2001;Helmert and Domshlak 2009;Richter and Westphal 2010;Helmert et al 2014;Domshlak, Hoffmann, and Katz 2015), training NNs as heuristic functions is highly promising, and is actively pursued (Toyer et al 2018;Garg, Bajpai, and Mausam 2019;Ferber, Helmert, and Hoffmann 2020;Shen, Trevizan, and Thiébaux 2020;Rivlin, Hazan, and Karpas 2020;Yu, Kuroiwa, and Fukunaga 2020;Karia and Srivastava 2021;Ferber et al 2022). We contribute a new angle to this line of research, honing in on NN prediction confidence.…”