Recognizing the Enemy: Combining Reinforcement Learning with Strategy Selection Using Case-Based Reasoning

Auslander, Bryan; Lee-Urban, Stephen; Hogg, Chad; Muñoz-Ávila, Héctor

doi:10.1007/978-3-540-85502-6_4

Cited by 36 publications

(32 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sharma et al [17] make use of CBR as a function approximator for RL, and RL as revision algorithm for CBR in a hybrid architecture system; Gabel and Riedmiller [18] also makes use of CBR in the task of approximating a function over high-dimensional, continuous spaces; Juell and Paulson [19] exploit the use of RL to learn similarity metrics in response to feedback from the environment; Auslander et al [20] use CBR to adapt quickly an RL agent to changing conditions of the environment by the use of previously stored policies and Li, Zonghai and Feng [21] propose an algorithm that makes use of knowledge acquired by reinforcement learning to construct and extend a case base. Finally, Bianchi, Ros and López de Mántaras [22] use CBR together with Heuristic Accelerated Reinforcement Learning to improve reinforcement learning by using case based heuristics.…”

Section: Transfer Learningmentioning

confidence: 99%

Using Transfer Learning to Speed-Up Reinforcement Learning: A Cased-Based Approach

Celiberto

Matsuura

Mántaras

et al. 2010

2010 Latin American Robotics Symposium and Intelligent Robotics Meeting

View full text Add to dashboard Cite

Abstract-Reinforcement Learning (RL) is a well known technique for the solution of problems where agents need to act with success in an unknown environment, learning through trial and error. However, this technique is not efficient enough to be used in applications with real world demands due to the time that the agent needs to learn. This paper investigates the use of Transfer Learning (TL) between agents to speed up the well known Q-learning Reinforcement Learning algorithm. The new approach presented here allows the use of cases in a case base as heuristics to speed up the Q-learning algorithm, combining Case-Based Reasoning (CBR) and Heuristically Accelerated Reinforcement Learning (HARL) techniques.A set of empirical evaluations were conducted in the Mountain Car Problem Domain, where the actions learned during the solution of the 2D version of the problem can be used to speed up the learning of the policies for its 3D version.The experiments were made comparing the Q-learning Reinforcement Learning algorithm, the HAQL Heuristic Accelerated Reinforcement Learning (HARL) algorithm and the TL-HAQL algorithm, proposed here. The results show that the use of a case-base for transfer learning can lead to a significant improvement in the performance of the agent, making it learn faster than using either RL or HARL methods alone.

show abstract

Section: Transfer Learningmentioning

confidence: 99%

Using Transfer Learning to Speed-Up Reinforcement Learning: A Cased-Based Approach

Celiberto

Matsuura

Mántaras

et al. 2010

2010 Latin American Robotics Symposium and Intelligent Robotics Meeting

View full text Add to dashboard Cite

show abstract

“…Two states are similar if the absolute difference of the attributes is smaller or equal than each of the corresponding entries in the table below. For example, (6,2,5,10) is similar to (3,1,8,5) relative to the major similarity but not relative to the minor similarity. The values in parenthesis in the Major similarity show the ranges for the large and small maps.…”

Section: Similarity Metricmentioning

confidence: 99%

“…The potential for integrating these two techniques has been demonstrated in a variety of domains including digital games [1] and robotics [2]. For the most part the integration has been aimed at exploiting synergies between RL and CBR that result in performance that is better than each individually (e.g., [3]) or to enhance the performance of the CBR system (e.g., [4]). Although researchers have pointed out that CBR could help to enhance RL processes [5], comparatively little research has been done in this direction, and the bulk of it has concentrated on tasks with continuous states [6,7,16,17].…”

Section: Introductionmentioning

confidence: 99%

Reducing the Memory Footprint of Temporal Difference Learning over Finitely Many States by Using Case-Based Generalization

Dilts

Muñoz-Ávila

2010

Case-Based Reasoning. Research and Development

Self Cite

View full text Add to dashboard Cite

Abstract. In this paper we present an approach for reducing the memory footprint requirement of temporal difference methods in which the set of states is finite. We use case-based generalization to group the states visited during the reinforcement learning process. We follow a lazy learning approach; cases are grouped in the order in which they are visited. Any new state visited is assigned to an existing entry in the Q-table provided that a similar state has been visited before. Otherwise a new entry is added to the Q-table. We performed experiments on a turn-based game where actions have non-deterministic effects and might have long term repercussions on the outcome of the game. The main conclusion from our experiments is that by using case-based generalization, the size of the Q-table can be substantially reduced while maintaining the quality of the RL estimates.

show abstract

“…RETALIATE (Reinforced Tactic Learning in Agent-Tam Environments), an online Q-Learning algorithm that creates strategies for teams of computer agents in the commercial First Person Shooter (FPS) game Unreal Tournament is introduced in [10]. This approach is extended in [11], where the authors use CBR in order to get the original RETALIATE algorithm to adapt more quickly to changes in the environment. IN COMPUTER GAME AI…”

Section: Related Workmentioning

confidence: 99%

Using reinforcement learning for city site selection in the turn-based strategy game Civilization IV

Wender

Watson

2008

2008 IEEE Symposium on Computational Intelligence and Games

View full text Add to dashboard Cite

Abstract-This paper describes the design and implementation of a reinforcement learner based on Q-Learning. This adaptive agent is applied to the city placement selection task in the commercial computer game Civilization IV. The city placement selection determines the founding sites for the cities in this turn-based empire building game from the Civilization series. Our aim is the creation of an adaptive machine learning approach for a task which is originally performed by a complex deterministic script. This machine learning approach results in a more challenging and dynamic computer AI. We present the preliminary findings on the performance of our reinforcement learning approach and we make a comparison between the performance of the adaptive agent and the original static game AI. Both the comparison and the performance measurements show encouraging results. Furthermore the behaviour and performance of the learning algorithm are elaborated and ways of extending our work are discussed.

show abstract

Recognizing the Enemy: Combining Reinforcement Learning with Strategy Selection Using Case-Based Reasoning

Cited by 36 publications

References 11 publications

Using Transfer Learning to Speed-Up Reinforcement Learning: A Cased-Based Approach

Using Transfer Learning to Speed-Up Reinforcement Learning: A Cased-Based Approach

Reducing the Memory Footprint of Temporal Difference Learning over Finitely Many States by Using Case-Based Generalization

Using reinforcement learning for city site selection in the turn-based strategy game Civilization IV

Contact Info

Product

Resources

About