No abstract

-One of the challenges of rescue robotics is to create robots that can autonomously traverse rough, unstructured terrain. Although mechanical engineering can produce very capable robots, mechanical engineering alone will not drive them. In this paper, we present a terrain feature extractor that can be taught to find significant features in range images of terrain around a robot from a human expert. This novel approach has the advantage that it potentially allows the human expert's knowledge to be captured rapidly. A terrain model is generated from the many points in the range sensor data. Techniques from the field of knowledge acquisition are then used to find patterns in the terrain model. A knowledge acquisition system can then be taught to drive a robot in unstructured terrain based on these features. We evaluate the performance of the initial stages of the feature extractor on a real robot, traversing NIST specification red stepfields.

No abstract

A challenge in applying reinforcement learning to large problems is how to manage the explosive increase in storage and time complexity. This is especially problematic in multi-agent systems, where the state space grows exponentially in the number of agents. Function approximation based on simple supervised learning is unlikely to scale to complex domains on its own, but structural abstraction that exploits system properties and problem representations shows more promise. In this paper, we investigate several classes of known abstractions: 1) symmetry, 2) decomposition into multiple agents, 3) hierarchical decomposition, and 4) sequential execution. We compare memory requirements, learning time, and solution quality empirically in two problem variations. Our results indicate that the most effective solutions come from combinations of structural abstractions, and encourage development of methods for automatic discovery in novel problem formulations.

Abstract. HEXQ is a reinforcement learning algorithm that discovers hierarchical structure automatically. The generated task hierarchy represents the problem at different levels of abstraction. In this paper we extend HEXQ with heuristics that automatically approximate the structure of the task hierarchy. Construction, learning and execution time, as well as storage requirements of a task hierarchy may be significantly reduced and traded off against solution quality.