Keep focussing: striatal dopamine multiple functions resolved in a single mechanism tested in a simulated humanoid robot

Abstract

The effects of striatal dopamine (DA) on behavior have been widely investigated over the past decades, with “phasic” burst firings considered as the key expression of a reward prediction error responsible for reinforcement learning. Less well studied is “tonic” DA, where putative functions include the idea that it is a regulator of vigor, incentive salience, disposition to exert an effort and a modulator of approach strategies. We present a model combining tonic and phasic DA to show how different outflows triggered by either intrinsically or extrinsically motivating stimuli dynamically affect the basal ganglia by impacting on a selection process this system performs on its cortical input. The model, which has been tested on the simulated humanoid robot iCub interacting with a mechatronic board, shows the putative functions ascribed to DA emerging from the combination of a standard computational mechanism coupled to a differential sensitivity to the presence of DA across the striatum.

A Multilevel Computational Characterization of Endophenotypes in Addiction

“…However, drug consumption triggers extra-physiologic dopamine-dependent learning, resulting in aberrantly high circuit gain, and compromising the ability of all affected circuits to discriminate among different inputs and produce temporal transitions towards multiple stable states. The cortico-striatal circuits become over-stable and resistant to perturbation caused by a change of input or by noise (Fiore et al, 2014) as they are dominated by parasitic attractors (Hoffman and McGlashan, 2001) (Fig 1C). In the ventral cortico-striatal circuit, a parasitic attractor sets and maintains the selection of drug-related goals or outcomes, biasing the action-outcome assessments required for planning.…”

“…The two models comprised a neural mass model that has been validated and described in the context of choice behavior and dopaminergic modulation (Baldassarre et al, 2013; Fiore et al, 2014; Fiore et al, 2016) and a normative or algorithmic model based upon standard RL schemes. In the neural model, addiction and treatment response were modeled through DA-dependent associative plasticity in both ventral and dorsal circuits.…”

Interplay of Rhythmic and Discrete Manipulation Movements During Development: A Policy-Search Reinforcement-Learning Robot Model

“…This is reminiscent of the brain organization illustrated in Section I for which discrete movements involve additional brain areas that might contribute to modulate the activity of areas generating rhythmic movements [2]. An architecture with a more biologically plausible structure might produce specific predictions on this topic, which might be tested against available or new empirical evidence (e.g., see [108], [109], [110], and [111]). This topic, however, deserves further investigation.…”

Ecological Active Vision: Four Bioinspired Principles to Integrate Bottom–Up and Adaptive Top–Down Attention Tested With a Simple Camera-Arm Robot

“…This would give enhanced properties to the current system, for example a staged and tunable integration of different information sources (here rigid). Moreover, it would allow a higher robustness to real-image noise as decisions would be taken by integrating information in time on the basis of a dynamic competition with parameters tunable on the fly as it happens in basal ganglia, rather than fixed as here [40]. In this respect, the amount of competition between competing bottom-up information sources, possibly based on parietal cortex (which represents an important source of input to superior colliculus and the basal ganglia areas controlling eye movements, [46]) has been suggested to play an important role in the development of attention [92].…”