A Robot in a Water Maze: Learning a Spatial Memory Task

Abstract

This paper explores several novel approaches to solve the Morris water maze task. In this spatial memory task, the robot must learn how to associate perceptual information with a particular location to aid in navigating to the goal. A Self-Organizing Feature Map (SOFM) is used to discretize the perceptual space. The robot must then learn to associate these perceptual states with an action used to navigate through the environment. Two navigational approaches are proposed. The first approach involves computing a probabilistic graph between SOFM nodes and then searching the graph to locate a path to the goal. The second approach uses temporal difference learning to learn the association between an SOFM node and an action that will direct it to the goal. The paper compares the effectiveness of these two approaches and discusses their respective utility.

Massively multi-robot simulation in stage

“…Yet, Stage has a presence at the major research conferences and on university course web pages that suggests it is widely used and useful. For example, at least one paper using Stage for multi-robot simulations has appeared at ICRA, the main annual robotics meeting, every year since its debut in 2001 (Ye et al 2001;Fredslund and Matarić 2002;Batalin and Sukhatme 2003;Shell and Matarić 2004;Batalin and Sukhatme 2004;Lin and Zheng 2005;Batalin and Sukhatme 2005;Chang et al 2006;Busch et al 2007). Only the first of these papers is by an author of Stage.…”

Adaptive temporal difference learning of spatial memory in the water maze task

“…A rectangular enclosure, modeled after the simulated environment from [6], is used for the "dry" water maze experiments. The enclosure is 5.26m by 6.06m, and contains a circular "hidden" platform that is 0.41m in diameter.…”

“…The enclosure is 5.26m by 6.06m, and contains a circular "hidden" platform that is 0.41m in diameter. Eighteen colored panels are arranged around the enclosure using approximately the same layout as in [6]. Fig.…”

“…Based on Krichmar's work, Busch et al [6] used a simulated water maze environment to compare an attributed probabilistic graph search approach and a temporal difference learning approach based solely on visual cues from the environment. The simulated robot was equipped with three cameras to gather perceptual information from the environment and used a Self-Organizing Map (SOM) [7] to discretize the perceptual space.…”

“…The work described in this paper is based on extending the temporal difference learning navigational approach used by Busch et al [6] from a simulated water maze environment into a physical "dry" water maze environment. Additionally, the adaptability of the temporal difference learning approach is investigated by moving the platform.…”