A Neural Network for Decision Making in Real-Time Heuristic Search

Abstract

Most real-time heuristic search algorithms solve search problems by executing a series of episodes. During each episode the algorithm decides an action for execution. Such a decision is usually made using information gathered by running a bounded, heuristic-search algorithm. In this paper we report on a real-time search algorithm that does not use a search algorithm to choose the next action to be applied. Rather, it uses a neural network whose input is local information about the search graph, comparable to the information that would be used by a bounded search algorithm. We describe a supervised learning approach to training such a network. Our three types of maps from the Moving AI benchmarks, shows that our algorithm is, in some cases, substantially superior to algorithms that have access to the same information about the graph. One of our most important conclusions is that our extended set of features important: indeed, using features beyond the heuristic seems key to achieving good performance.

Exploiting Learned Policies in Focal Search

“…In the past few years, machine learning (ML) approaches have been proposed to enhance the performance of AI search and domain-independent planning. A number of these approaches could be classified as learning a heuristic estimate (e.g., Yoon, Fern, and Givan 2006;Arfaee, Zilles, and Holte 2011;Thayer, Dionne, and Ruml 2011;Ferber, Helmert, and Hoffmann 2020), a policy (e.g., Groshev et al 2018;Muñoz et al 2018;Toyer et al 2020), or a combination of both (e.g., McAleer et al 2019).…”