Exploiting Learned Policies in Focal Search

Abstract

Recent machine-learning approaches to deterministic search and domain-independent planning employ policy learning to speed up search. Unfortunately, when attempting to solve a search problem by successively applying a policy, no guarantees can be given on solution quality. The problem of how to effectively use a learned policy within a bounded-suboptimal search algorithm remains largely as an open question. In this paper, we propose various ways in which such policies can be integrated into Focal Search, assuming that the policy is a neural network classifier. Furthermore, we provide mathematical foundations for some of the resulting algorithms. To evaluate the resulting algorithms over a number of policies with varying accuracy, we use synthetic policies which can be generated for a target accuracy for problems where the search space can be held in memory. We evaluate our focal search variants over three benchmark domains using our synthetic approach, and on the 15-puzzle using a neural network learned using 1.5 million examples. We observe that Discrepancy Focal Search, which we show expands the node which maximizes an approximation of the probability that its corresponding path is a prefix of an optimal path, obtains, in general, the best results in terms of runtime and solution quality.

Focal Discrepancy Search for Learned Heuristics (Extended Abstract)

“…More formally, if s is a state in FOCAL, at any point during the execution of FS, its priority is given by disc(path(s)). FDS has been proposed recently in the context of learned policies (Araneda, Greco, and Baier 2021), in which the notion of discrepancy is not defined in terms of a heuristic but rather in terms of a policy.…”

“…Spies et al (2019) proposed to use the learned heuristic to sort FOCAL. Araneda, Greco, and Baier (2021) proposed an approach to exploit learned policies, rather than a heuristic, in the context of FS. They show that an effective way to incorporate policies in FS is by exploiting the notion of discrepancy (Harvey and Ginsberg 1995), which given a stateaction sequence s 0 a 0 s 1 a 1 .…”

“…In this paper, strongly inspired by the work of Araneda, Greco, and Baier (2021), we propose to use Focal Discrepancy Search (FDS) in the context of bounded-suboptimal search with learned heuristics, which sorts FOCAL using a discrepancy score computed from the learned heuristic.…”

Avoiding Errors in Learned Heuristics in Bounded-Suboptimal Search

“…Focal Discrepancy Search (FDS) (Araneda, Greco, and Baier 2021;Greco, Araneda, and Baier 2022) is a version of Focal Search which sorts FOCAL by the discrepancy associated with the path of each state. More formally, if s is a state in FOCAL, at any point during the execution of FS, its priority is given by disc(path(s)), which is the number of times along the path where the state with the best heuristic value was not selected for expansion.…”

K-Focal Search for Slow Learned Heuristics (Extended Abstract)

“…BWAS does not provide suboptimality guarantees, as a consequence of a second drawback of neural-net heuristics: they are not admissible, thus they cannot be directly used by the many search algorithms that exploit admissibility to provide quality guarantees. Recently, Spies et al [2019] and Araneda, Greco, and Baier;Greco, Araneda, and Baier [2021;2022] proposed the use of neural-net heuristics in combination with admissible heuristics in Focal Search (FS). However, these approaches do not address the problem of slow heuristic computation.…”