Abstract. This paper presents CBRetaliate, an agent that combines Case-Based Reasoning (CBR) and Reinforcement Learning (RL) algorithms. Unlike most previous work where RL is used to improve accuracy in the action selection process, CBRetaliate uses CBR to allow RL to respond more quickly to changing conditions. CBRetaliate combines two key features: it uses a time window to compute similarity and stores and reuses complete Q-tables for continuous problem solving. We demonstrate CBRetaliate on a team-based first-person shooter game, where our combined CBR+RL approach adapts quicker to changing tactics by an opponent than standalone RL.

We describe HTN-Maker, an algorithm for learning hierarchical planning knowledge in the form of task-reduction methods for Hierarchical Task Networks (HTNs). HTN-Maker takes as input a set of planning states from a classical planning domain and plans that are applicable to those states, as well as a set of semantically-annotated tasks to be accomplished. The algorithm analyzes this semantic information in order to determine which portion of the input plans accomplishes a particular task and constructs task-reduction methods based on those analyses.We present theoretical results showing that HTN-Maker is sound and complete. Our experiments in five well-known planning domains confirm the theoretical results and demonstrate convergence toward a set of HTN methods that can be used to solve any problem expressible as a classical planning problem in that domain, relative to a set of goal types for which tasks have been defined. In three of the five domains, HTN planning with the learned methods scales much better than a modern classical planner.

We consider how to learn Hierarchical Task Networks (HTNs) for planning problems in which both the quality of solution plans generated by the HTNs and the speed at which those plans are found is important. We describe an integration of HTN Learning with Reinforcement Learning to both learn methods by analyzing semantic annotations on tasks and to produce estimates of the expected values of the learned methods by performing Monte Carlo updates. We performed an experiment in which plan quality was inversely related to plan length. In two planning domains, we evaluated the planning performance of the learned methods in comparison to two state-of-the-art satisficing classical planners, FastForward and SGPlan6, and one optimal planner, HSP*. The results demonstrate that a greedy HTN planner using the learned methods was able to generate higher quality solutions than SGPlan6 in both domains and FastForward in one. Our planner, FastForward, and SGPlan6 ran in similar time, while HSP* was exponentially slower.

No abstract

Abstract. This paper presents a new approach for spatial event prediction that combines a value function approximation algorithm and case-based reasoning predictors. Each of these predictors makes unique contributions to the overall spatial event prediction. The function value approximation prediction is particularly suitable to reasoning with geographical features such as the (x,y) coordinates of an event. The case-based prediction is particularly well suited to deal with non-geographical features such as the time of the event or income level of the population. We claim that the combination of these two predictors results in a significant improvement of the accuracy in the spatial event prediction compared to pure geographically-based predictions. We support our claim by reporting on an ablation study for the prediction of improvised explosive device (IED) attacks.