Humans and animals take actions quickly when they expect that the actions lead to reward, reflecting their motivation. Injection of dopamine receptor antagonists into the striatum has been shown to slow such reward-seeking behavior, suggesting that dopamine is involved in the control of motivational processes. Meanwhile, neurophysiological studies have revealed that phasic response of dopamine neurons appears to represent reward prediction error, indicating that dopamine plays central roles in reinforcement learning. However, previous attempts to elucidate the mechanisms of these dopaminergic controls have not fully explained how the motivational and learning aspects are related and whether they can be understood by the way the activity of dopamine neurons itself is controlled by their upstream circuitries. To address this issue, we constructed a closed-circuit model of the corticobasal ganglia system based on recent findings regarding intracortical and corticostriatal circuit architectures. Simulations show that the model could reproduce the observed distinct motivational effects of D 1 -and D 2 -type dopamine receptor antagonists. Simultaneously, our model successfully explains the dopaminergic representation of reward prediction error as observed in behaving animals during learning tasks and could also explain distinct choice biases induced by optogenetic stimulation of the D 1 and D 2 receptor-expressing striatal neurons. These results indicate that the suggested roles of dopamine in motivational control and reinforcement learning can be understood in a unified manner through a notion that the indirect pathway of the basal ganglia represents the value of states/actions at a previous time point, an empirically driven key assumption of our model.
IntroductionDopamine has been suggested to control motivation and rewardseeking behavior (Robbins and Everitt, 1996;Berridge and Robinson, 1998;Dayan and Balleine, 2002; McClure et al., 2003;. As a direct evidence, application of dopamine receptor antagonists in the striatum has been shown to slow the subject's behavior (Salamone and Correa, 2002; Nakamura and Hikosaka, 2006), with distinct effects observed for the antagonists of D1-and D2-type dopamine receptors (D1Rs and D2Rs), which are expressed in distinct populations of striatal medium spiny neurons (MSNs) projecting to the "direct" (dMSNs) and "indirect" (iMSNs) pathways of the basal ganglia, respectively (Gerfen and Surmeier, 2011). Clarifying mechanisms of such pharmacological effects is likely to help elucidating the exact roles of dopamine in motivational control, and neural circuit modeling has been used as one of the powerful approaches (Frank et al., 2004;. However, these models are still not self-contained in a sense that responses of either the dopamine neurons or their hypothesized upstream globus pallidus (GP) neurons were presumed rather than explained by inputs from the rest part of the circuit.Along with the suggested roles in motivational control, dopamine has also been suggested to be centrally...