Analogical Reinforcement Learning

Abstract

Research in analogical reasoning suggests that higher-order cognitive functions such as abstract reasoning, far transfer, and creativity are founded on recognizing structural similarities among relational systems. Here we integrate theories of analogy with the computational framework of reinforcement learning (RL). We propose a computational synergy between analogy and RL, in which analogical comparison provides the RL learning algorithm with a measure of relational similarity, and RL provides feedback signals that can drive analogical learning. Initial simulation results support the power of this approach.

Reinforcement Learning with Analogical Similarity to Guide Schema Induction and Attention

“…Attention learning is an additional mechanism of the model whose purpose is to increase the model's reliance on useful exemplars. Although the model makes sense without this attention learning mechanism, including it improves performance and integrates analogy, RL, and attention, and has been demonstrated in experiments with humans (Foster and Jones, 2013b;Foster, 2015). A model that increases its repertoire of concepts needs some pruning mechanism to sort through what's been discovered.…”

“…The analogical RL model was tested on its ability to learn tic-tac-toe (Foster and Jones, 2013a). Tic-tac-toe was chosen as a test domain because it is a simple game with relational structure and a clear task goal.…”