Affect, Anticipation, and Adaptation: Affect-Controlled Selection of Anticipatory Simulation in Artificial Adaptive Agents

Abstract

Emotion plays an important role in thinking. In this article we study affective control of the amount of simulated anticipatory behavior in adaptive agents using a computational model. Our approach is based on model-based reinforcement learning (RL) and inspired by the simulation hypothesis (Cotterill, 2001;Hesslow, 2002). The simulation hypothesis states that thinking is internal simulation of behavior using the same sensory-motor systems as those used for overt behavior. Here, we study the adaptiveness of an artificial agent, when action-selection bias is induced by an affect-controlled amount of simulated anticipatory behavior. To this end, we introduce an affect-controlled simulation-selection mechanism that uses the predictions of the agent's RL model to select anticipatory behaviors for simulation. Based on experiments with adaptive agents in two nondeterministic partially observable gridworlds we conclude that (1) internal simulation has an adaptive benefit and (2) affective control can reduce the amount of simulation needed for this benefit. This is specifically the case if the following relation holds: positive affect decreases the amount of simulation towards simulating the best potential next action, while negative affect increases the amount of simulation towards simulating all potential next actions. In essence we use artificial affect to control mental exploration versus exploitations. Thus, agents "feeling positive" can think ahead in a narrow sense and free up working memory resources, while agents "feeling negative" must think ahead in a broad sense and maximize usage of working memory. Our results are consistent with several psychological findings on the relation between affect and learning, and contribute to answering the question of when positive versus negative affect is useful during adaptation.

Association of Cognitive, Affective, Psychomotor and Intuitive Domains in Education, Sönmez Model

“…Based on this data it can be said that there is a high level of positive significant correlation between cognitive-psychomotor learning outcomes and affective learning outcomes. The previous research also support this finding [1,5,7,12,19,25]. Also a significant association was found between cognitive domain and attitude and motivation [7, 13, 14, 16, 18, 20, 22].…”

The Neuromodulatory System: A Framework for Survival and Adaptive Behavior in a Challenging World

“…In an interesting model that combines emotional affect with internal anticipation, Broekens and colleagues modeled action selection through internal simulation of behavior (Broekens, Kosters, & Verbeek, 2007). The learning by their agents was influenced by affect: the positiveness and negativeness of a situation.…”

“…It will be of interest to test this network model based on the architecture in Figure 1 in a more diverse environment. For example, many of the dynamic environments from the artificial intelligence and autonomous agent community may provide a challenging test bed for a neurobiologically inspired model, as well as an interesting comparison of approaches (Avila-Garcia & Cañamero, 2004;Broekens et al, 2007;Cañamero, 1997;Prescott et al, 2006).…”

Classification of PISA 2012 Mathematical Literacy Scores Using Decision-Tree Method: Turkey Sampling

“…Bu durum, öğrenmenin duyuşsal boyutunun daha fazla önem taşımaya başladığını göstermektedir (Lehman, 2006). Duyguların hem düşüncelerimizi hem de davranışlarımızı etkilediği düşünüldüğünde duyuşsal özelliklerin öğrenme ortamındaki rolünün oldukça büyük olduğu düşünülmektedir (Picard, 1997;Broekens, Kosters ve Verbeek, 2007). Öğrencilerin matematik dersinde başarılı ya da başarısız olmalarında duyuşsal hazır oluşlarının rolü oldukça büyüktür (Çoban, 1989).…”