A neurorobotic platform to test the influence of neuromodulatory signaling on anxious and curious behavior

Abstract

The vertebrate neuromodulatory systems are critical for appropriate value-laden responses to environmental challenges. Whereas changes in the overall level of dopamine (DA) have an effect on the organism's reward or curiosity-seeking behavior, changes in the level of serotonin (5-HT) can affect its level of anxiety or harm aversion. Moreover, top-down signals from frontal cortex can exert cognitive control on these neuromodulatory systems. The cholinergic (ACh) and noradrenergic (NE) systems affect the ability to filter out noise and irrelevant events. We introduce a neural network for action selection that is based on these principles of neuromodulatory systems. The algorithm tested the hypothesis that high levels of serotonin lead to withdrawn behavior by suppressing DA action and that high levels of DA or low levels of 5-HT lead to curious, exploratory behavior. Furthermore, the algorithm tested the idea that top-down signals from the frontal cortex to neuromodulatory areas are critical for an organism to cope with both stressful and novel events. The neural network was implemented on an autonomous robot and tested in an open-field paradigm. The open-field test is often used to test for models anxiety or exploratory behavior in the rodent and allows for qualitative comparisons with the neurorobot's behavior. The present neurorobotic experiments can lead to a better understanding of how neuromodulatory signaling affects the balance between anxious and curious behavior. Therefore, this experimental paradigm may also be informative in exploring a wide range of neurological diseases such as anxiety, autism, attention deficit disorders, and obsessive-compulsive disorders.

Hedonic quality or reward? A study of basic pleasure in homeostasis and decision making of a motivated autonomous robot

“…In previous work, we have investigated the role of pleasure as reward in the maintenance of homeostasis in a reinforcement learning model (Cos et al, 2013). In this paper, we depart from the idea that pleasure is necessarily linked with reward—in the same way as value is not necessarily linked with reward (Krichmar & Röhrbein, 2013)—or with signaling biological usefulness, opening the door to the investigation of the role of other types of pleasure not directly related with the satisfaction of needs (Frijda, 2010), in addition to pleasure stemming from need satisfaction.…”

“…In both cases, we have focused our study on the influence that pleasure has on the perception of external stimuli, and the underlying mechanism we have adopted to model pleasure is a simulation of hormonal modulation of perception. Unlike related work in robotics (Krichmar, 2008, 2012, 2013; Sporns & Alexander, 2002), our simulated hormone constitutes an abstract model aimed to capture the (gross) dynamics of modulation rather than modeling the behavior of specific chemicals underlying pleasure and more generally affective phenomena. Hormonal modulation has been used in robots for other purposes, such as behavior control (Moioli, Vargas, & Husbands, 2009), learning based on value and reward systems (e.g.…”

“…Hormonal modulation has been used in robots for other purposes, such as behavior control (Moioli, Vargas, & Husbands, 2009), learning based on value and reward systems (e.g. the 2013 special issue of Frontiers in Neurorobotics on this topic; see (Krichmar and Röhrbein, 2013) for a review of the papers), to model intrinsic motivation (Kaplan & Oudeyer, 2007; Luciw, Kompella, Kazerounian, & Schmidhuber, 2013), energetic autonomy (Lowe et al, 2010; Sauzé & Neal, 2010), and evolutionary and modular robotics (Hamann, Stradner, Schmickl, & Crailsheim, 2010). …”

Nicotine and methamphetamine disrupt habituation of sensory reinforcer effectiveness in male rats.

“…According to the reward error signal hypothesis, unexpected sensory stimuli that predict important biological reinforcers evoke phasic DA responses which reflect the expected amount of reward. Bolado-Gomez and Gurney (2013) suggest that the sensory prediction error signal is modulated by reward value. For example, the phasic firing evoked by response-contingent light- onset normally habituates, however this habituation is prevented if light-onset predicts a biologically important reinforcer.…”

Value and reward based learning in neurorobots

“…Their model was inspired by cortical regions of the brain involved in unsupervised learning, as well as neuromodulatory systems responsible for providing intrinsic rewards through dopamine and regulating levels of attention through norepinephrine. Different neuromodulatory systems in the brain may be related to different aspects of value (Krichmar, 2013). In a model of multiple neuromodulatory systems, Krichmar showed that interactions between the dopaminergic (reward), serotoninergic (harm aversion), and the cholinergic/noradrenergic (novelty) systems could lead to interesting behavioral control in an autonomous robot.…”