Modeling Behavior Cycles as a Value System for Developmental Robots

Abstract

The behavior of natural systems is governed by rhythmic behavior cycles at the biological, cognitive, and social levels. These cycles permit natural organisms to adapt their behavior to their environment for survival, behavioral efficiency, or evolutionary advantage. This article proposes a model of behavior cycles as the basis for motivated reinforcement learning in developmental robots. Motivated reinforcement learning is a machine learning technique that incorporates a value system with a trial-anderror learning component. Motivated reinforcement learning is a promising model for developmental robotics because it provides a way for artificial agents to build and adapt their skill-sets autonomously over time. However, new models and metrics are needed to scale existing motivated reinforcement learning algorithms to the complex, real-world environments inhabited by robots. This article presents two such models and an experimental evaluation on four Lego Mindstorms NXT robots. Results show that the robots can evolve measurable, structured behavior cycles adapted to their individual physical forms.

Self-motivated learning of achievement and maintenance tasks for non-player characters in computer games

“…3) Competence Merrick [19] proposes the following signal to motivate the acquisition of competence: (4) According to this motivation signal, highly motivating tasks must be repeatable and repetition must cause the agent to learn.…”

Hedonic value: enhancing adaptation for motivated agents

“…Previous studies have investigated several principles related to the notion of value to devise increasingly flexible strategies of adaptation for mobile robotics: behavioral cycles to interact with the environment (Ahlgren & Halberg, 1990; McFarland & Spier, 1997), new algorithms to incorporate novelty (Huang & Weng, 2002), hierarchical RL architectures for skill learning (Baldassarre, 2002; G. Konidaris, Kuindersma, Barto, & Grupen, 2010), or the use of algorithms to select learning goals autonomously using a value system in a RL context (Merrick, 2010). The operation of each of these approaches is to some extent based on exploiting the interaction with the environment as a guide to structure behavioral responses.…”

A Temporal Difference GNG-Based Algorithm That Can Learn to Control in Reinforcement Learning Environments

“…This type of learning has been extensively used with success in many problems where the expected behavior is not known [2], [3]. These algorithms, however, suffers from the curse of dimensionality [4] where the search space grows exponentially as the number of states and actions increases [5].…”