Dopamine, Affordance and Active Inference

Abstract

The role of dopamine in behaviour and decision-making is often cast in terms of reinforcement learning and optimal decision theory. Here, we present an alternative view that frames the physiology of dopamine in terms of Bayes-optimal behaviour. In this account, dopamine controls the precision or salience of (external or internal) cues that engender action. In other words, dopamine balances bottom-up sensory information and top-down prior beliefs when making hierarchical inferences (predictions) about cues that have affordance. In this paper, we focus on the consequences of changing tonic levels of dopamine firing using simulations of cued sequential movements. Crucially, the predictions driving movements are based upon a hierarchical generative model that infers the context in which movements are made. This means that we can confuse agents by changing the context (order) in which cues are presented. These simulations provide a (Bayes-optimal) model of contextual uncertainty and set switching that can be quantified in terms of behavioural and electrophysiological responses. Furthermore, one can simulate dopaminergic lesions (by changing the precision of prediction errors) to produce pathological behaviours that are reminiscent of those seen in neurological disorders such as Parkinson's disease. We use these simulations to demonstrate how a single functional role for dopamine at the synaptic level can manifest in different ways at the behavioural level.

Action Reprogramming in Parkinson's Disease: Response to Prediction Error Is Modulated by Levels of Dopamine

“…The latter result implies that the behavioral consequences of a prediction error, i.e., increased RT during AR, can be modulated by dopamine, which is strikingly similar to the theoretical consequences of the "active inference" framework of Friston et al (2010Friston et al ( , 2012. This proposes that actions are performed in response to sensory prediction errors and that dopamine levels encode the reliability or precision of sensory information.…”

“…Within this framework, dopamine is important for balancing bottom-up sensory information with top-down prior beliefs when making inferences about sensory cues that have affordance. In other words, dopamine encodes the value of the sensory prediction error (Krugel et al, 2009;Friston et al, 2010Friston et al, , 2012. Therefore, when we experience unexpected sensory information with high precision, we are prepared to make fast corrections of the prepotent but incorrect action.…”

Model-Based Approaches to Active Perception and Control

“…For example, an agent's generative model can be composed of a simple dynamical system [117,118] (e.g., a pendulum) that guides the active sampling of information, in analogy to rodent whisking behaviour. In this example, the pendulum may jointly support the control of a simple whisker-like sensor and the prediction of a sensory event following its protraction (with a certain amplitude)-or a sensorimotor contingency between whisker protraction and the receipt of sensory stimuli.…”

Hypotheses Relating to the Function of the Claustrum

“…It is possible, therefore, that the claustrum might perform both functions described above-computing binding and computing global reinforcement-in different locations. The brain contains several systems involved in salience detection-for example the mesolimbic dopamine system (Enomoto et al, 2011;Friston et al, 2012). Another salience network includes the dorsal anterior cingulate cortex, middle and inferior temporal cortex and the fronto-insular cortex (Cauda et al, 2011;Yuan et al, 2012).…”