Neural Network Heuristic Functions: Taking Confidence into Account

Abstract

Neural networks (NN) are increasingly investigated in AI Planning, and are used successfully to learn heuristic functions. NNs commonly not only predict a value, but also output a confidence in this prediction. From the perspective of heuristic search with NN heuristics, it is a natural idea to take this into account, e.g. falling back to a standard heuristic where confidence is low. We contribute an empirical study of this idea. We design search methods which prune nodes, or switch between search queues, based on the confidence of NNs. We furthermore explore the possibility of out-of-distribution (OOD) training, which tries to reduce the overconfidence of NNs on inputs different to the training distribution. In experiments on IPC benchmarks, we find that our search methods improve coverage over standard methods, and that OOD training has the desired effect in terms of prediction accuracy and confidence, though its impact on search seems marginal.

Searching with Distributional Heuristics (Student Abstract)

“…The problem of online generation of distributional heuristics has received little attention, and this only recently. Though I am more interested in online learning, the approach by Heller et al (2022) of considering confidence when selecting nodes for expansion is promising. In the bounded cost heuristic search setting, Fickert, Gu, and Ruml (2021) synthesized a Gaussian distributional heuristic with a variance based on an estimate of single-step error learned online, and a mean based on the error-corrected value f .…”