Strategies for Affect-Controlled Action-Selection in Soar-RL

“…Second, Soar allows many forms of planning, enabling a better comparison between affective control of planning versus forward internal simulation. We are currently investigating affect-based control techniques in Soar-RL (Hogewoning, Broekens, Eggermont, & Bovenkamp, 2007).…”

Section: Future Workmentioning

confidence: 99%

Affect, Anticipation, and Adaptation: Affect-Controlled Selection of Anticipatory Simulation in Artificial Adaptive Agents

2007

Self Cite

View full text Add to dashboard Cite

Emotion plays an important role in thinking. In this article we study affective control of the amount of simulated anticipatory behavior in adaptive agents using a computational model. Our approach is based on model-based reinforcement learning (RL) and inspired by the simulation hypothesis (Cotterill, 2001;Hesslow, 2002). The simulation hypothesis states that thinking is internal simulation of behavior using the same sensory-motor systems as those used for overt behavior. Here, we study the adaptiveness of an artificial agent, when action-selection bias is induced by an affect-controlled amount of simulated anticipatory behavior. To this end, we introduce an affect-controlled simulation-selection mechanism that uses the predictions of the agent's RL model to select anticipatory behaviors for simulation. Based on experiments with adaptive agents in two nondeterministic partially observable gridworlds we conclude that (1) internal simulation has an adaptive benefit and (2) affective control can reduce the amount of simulation needed for this benefit. This is specifically the case if the following relation holds: positive affect decreases the amount of simulation towards simulating the best potential next action, while negative affect increases the amount of simulation towards simulating all potential next actions. In essence we use artificial affect to control mental exploration versus exploitations. Thus, agents "feeling positive" can think ahead in a narrow sense and free up working memory resources, while agents "feeling negative" must think ahead in a broad sense and maximize usage of working memory. Our results are consistent with several psychological findings on the relation between affect and learning, and contribute to answering the question of when positive versus negative affect is useful during adaptation.

show abstract

“…Connections of emotions with reinforcement learning have been explored by a number of authors (Moerland et al, 2017;Hogewoning et al, 2007;Marinier III and Laird, 2008;El-Nasr et al, 2000;Broekens et al, 2015). In traditional reinforcement learning (RL), an agent is tasked with both learning about his world (primarily the utility of situations), and of acting in the same world.…”

Section: Artificial Intelligencementioning

confidence: 99%

“…Exploration, on the other hand, is seen as something that could be guided by any number of (possibly affective) elements. For example, in (Hogewoning et al, 2007), higher valence (which is equivalent to reward being higher than expected, so things are going well) is used to push an agent to increased exploitation of current knowledge, whereas lower valence/reward (things are going worse than expected) pushes an agent to explore. This view is largely consistent with the affect-as-cognitive-feedback view (Huntsinger et al, 2014) where higher valence facilitates usage of existing mental constructs, whereas negative valence inhibits (and thus forces an agent to seek new solutions through exploration).…”

Section: Artificial Intelligencementioning

confidence: 99%

Structure is Management of Uncertainty in Groups

Hoey¹

2020

Preprint

View full text Add to dashboard Cite

I argue that the management of uncertainty by agents in a social world is foundational to the formation of social structures and to the definition of culture. I present a deep Bayesian model for this management of uncertainty in intelligent systems, and I argue for its applicability to cultural sociology. As social systems grow more heterogeneous, management of uncertainty in any participating agent becomes computationally difficult, and I propose that combinations of a small number of layers of reasoning in a deep Bayesian model are sufficient to account for some of the salient ways by which humans manage this uncertainty. Three forces come into play when considering such a model, and each is connected to a particular form of uncertainty. A denotative layer in the model represents uncertainty in the world or environment (ambiguity and risk about outcomes), a connotative layer manages the uncertainty about relationships with other social agents, and the connection between denotative and connotative handles uncertainty about identities of the self and others, and of behaviours taken. I show how the tradeoff between these three factors maps to different social structures, and I use use the model to make predictions across a range of domains, and show its relationship to cultural sociological, social psychological, economic and sociological theorizing. I further link this model to Bayesian views of the mind, primarily the enactive inference model of human intelligence, and compare and contrast to more traditional artificial intelligence.

show abstract

Strategies for Affect-Controlled Action-Selection in Soar-RL

Cited by 12 publications

References 5 publications

Social Human-Robot Interaction: A New Cognitive and Affective Interaction-Oriented Architecture

Social Human-Robot Interaction: A New Cognitive and Affective Interaction-Oriented Architecture

Affect, Anticipation, and Adaptation: Affect-Controlled Selection of Anticipatory Simulation in Artificial Adaptive Agents

Structure is Management of Uncertainty in Groups

Contact Info

Product

Resources

About