Anxious Learning in Real-Time Heuristic Search

Bulitko, Vadim; Doucet, Kacy

doi:10.1109/cig.2018.8490400

Cited by 6 publications

(14 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second way of thinking about the performance degradation lies with the notion of heuristic depressions (Ishida 1997) which has been subject of recent research (Hernández and Baier 2014;Sturtevant and Bulitko 2016;Bulitko and Sampley 2016). Viewed in terms of heuristic depressions, the degradation of LRTA* with self-knowledge can be viewed as follows.…”

Section: Counter Examplementioning

confidence: 99%

Effects of Self-Knowledge: Once Bitten Twice Shy

Bulitko

2021

AIIDE

View full text Add to dashboard Cite

Procedurally generating rich, naturally behaving AI-controlled video game characters is an important open problem. In this paper we focus on a particular aspect of non-playable character (NPC) behavior, long favored by science-fiction writers. Specifically, we study the effects of self-knowledge on NPC behavior. To do so we adopt the well-known framework of agent-centered real-time heuristic search applied to the standard pathfinding task on video-game maps. Such search agents normally use a heuristic function to guide them around a map to the goal state. Heuristic functions are inaccurate underestimates of the remaining distance to goal. What if the agent somehow knew how long it (the agent) would actually take to reach the goal from each state? How would using such self-knowledge in place of a heuristic function affect the agent's behavior? We show that similarly to real life, knowing of one's irrational behavior in a situation can deter the agent from getting into that situation again even if it is, in fact, a part of an optimal solution. We demonstrate the "fear" with a simple example and empirically show that the issue is common in video-game pathfinding. We then analyze the issue theoretically and suggest that "fear" induced by self-knowledge is not a bug but a feature and may potentially be used to develop more naturally behaving NPCs.

show abstract

Section: Counter Examplementioning

confidence: 99%

Effects of Self-Knowledge: Once Bitten Twice Shy

Bulitko

2021

AIIDE

View full text Add to dashboard Cite

show abstract

“…We assume that: (A1) action costs are greater than 0, (A2) for every state, there is a goal reachable from it, (A3) all initial beliefs have finite expected value, (A4) the state space is finite. Our proof follows the style of Korf's (1990) proof for RTA* and Bulitko and Sampley's (2016) proof for Weighted Lateral LRTA* (wbLRTA*): we first prove that incompleteness implies that there must exist a subset of states within which Nancy circulates forever. Then we prove that there cannot exist such a set due to the updates made by the learning rule of Nancy.…”

Section: Theoretical Analysismentioning

confidence: 99%

Beliefs We Can Believe in: Replacing Assumptions with Data in Real-Time Search

Fickert

Staut

et al. 2020

AAAI

View full text Add to dashboard Cite

Suboptimal heuristic search algorithms can benefit from reasoning about heuristic error, especially in a real-time setting where there is not enough time to search all the way to a goal. However, current reasoning methods implicitly or explicitly incorporate assumptions about the cost-to-go function. We consider a recent real-time search algorithm, called Nancy, that manipulates explicit beliefs about the cost-to-go. The original presentation of Nancy assumed that these beliefs are Gaussian, with parameters following a certain form. In this paper, we explore how to replace these assumptions with actual data. We develop a data-driven variant of Nancy, DDNancy, that bases its beliefs on heuristic performance statistics from the same domain. We extend Nancy and DDNancy with the notion of persistence and prove their completeness. Experimental results show that DDNancy can perform well in domains in which the original assumption-based Nancy performs poorly.

show abstract

“…LRTA* originally updated the heuristic value of a state based on the surrounding state with the minimum heuristic estimate to the goal. Different heuristic learning rules have also been researched for alternatives to the original methods used in LRTA* (Bulitko 2016d;2016a). Weighting the learning of the agent has been explored and shown to help speed up converging to the correct heuristic value (Shimbo and Ishida 2003;Rivera, Baier, and Hernndez 2015).…”

Section: Introduction and Related Workmentioning

confidence: 99%

Deep Learning for Real-Time Heuristic Search Algorithm Selection

Sigurdson

Bulitko

2021

AIIDE

View full text Add to dashboard Cite

Real-time heuristic search algorithms are used for creating agents that rely on local information and move in a bounded amount of time making them an excellent candidate for video games as planning time can be controlled. Path finding on video game maps has become the de facto standard for evaluating real-time heuristic search algorithms. Over the years researchers have worked to identify areas where these algorithms perform poorly in an attempt to mitigate their weaknesses. Recent work illustrates the benefits of tailoring algorithms for a given problem as performance is heavily dependent on the search space. In order to determine which algorithm to select for solving the search problems on a map the developer would have to run all the algorithms in consideration to obtain the correct choice. Our work extends the previous algorithm selection approach to use a deep learning classifier to select the algorithm to use on new maps without having to evaluate the algorithms on the map. To do so we select a portfolio of algorithms and train a classifier to predict which portfolio member to use on a unseen new map. Our empirical results show that selecting algorithms dynamically can outperform the single best algorithm from the portfolio on new maps, as well provide the lower bound for potential improvements to motivate further work on this approach.

show abstract

Anxious Learning in Real-Time Heuristic Search

Cited by 6 publications

References 19 publications

Effects of Self-Knowledge: Once Bitten Twice Shy

Effects of Self-Knowledge: Once Bitten Twice Shy

Beliefs We Can Believe in: Replacing Assumptions with Data in Real-Time Search

Deep Learning for Real-Time Heuristic Search Algorithm Selection

Contact Info

Product

Resources

About