Discrete coding of stimulus value, reward expectation, and reward prediction error in the dorsal striatum

Oyama, Kei; Tateyama, Yukina; Hernádi, István; Tobler, Philippe N.; Iijima, Toshio; Tsutsui, Ken

doi:10.1152/jn.00097.2015

Cited by 27 publications

(25 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The activity in the reward period remained unaltered even in omission trials. Similar neuronal signals have been reported in the striatum in mammals (Tremblay et al, 1998; Kim et al, 2009; Oyama et al, 2015). Note, however, that in the study by Kim et al (2009), the researchers expected the neurons to code the action value rather than the state value.…”

Section: Discussionsupporting

confidence: 82%

Striatal and Tegmental Neurons Code Critical Signals for Temporal-Difference Learning of State Value in Domestic Chicks

2016

View full text Add to dashboard Cite

To ensure survival, animals must update the internal representations of their environment in a trial-and-error fashion. Psychological studies of associative learning and neurophysiological analyses of dopaminergic neurons have suggested that this updating process involves the temporal-difference (TD) method in the basal ganglia network. However, the way in which the component variables of the TD method are implemented at the neuronal level is unclear. To investigate the underlying neural mechanisms, we trained domestic chicks to associate color cues with food rewards. We recorded neuronal activities from the medial striatum or tegmentum in a freely behaving condition and examined how reward omission changed neuronal firing. To compare neuronal activities with the signals assumed in the TD method, we simulated the behavioral task in the form of a finite sequence composed of discrete steps of time. The three signals assumed in the simulated task were the prediction signal, the target signal for updating, and the TD-error signal. In both the medial striatum and tegmentum, the majority of recorded neurons were categorized into three types according to their fitness for three models, though these neurons tended to form a continuum spectrum without distinct differences in the firing rate. Specifically, two types of striatal neurons successfully mimicked the target signal and the prediction signal. A linear summation of these two types of striatum neurons was a good fit for the activity of one type of tegmental neurons mimicking the TD-error signal. The present study thus demonstrates that the striatum and tegmentum can convey the signals critically required for the TD method. Based on the theoretical and neurophysiological studies, together with tract-tracing data, we propose a novel model to explain how the convergence of signals represented in the striatum could lead to the computation of TD error in tegmental dopaminergic neurons.

show abstract

Section: Discussionsupporting

confidence: 82%

Striatal and Tegmental Neurons Code Critical Signals for Temporal-Difference Learning of State Value in Domestic Chicks

2016

View full text Add to dashboard Cite

show abstract

“…Whereas some previous work report activity in dorsal striatum to be unaffected by cue onset (Berke, ; Kimchi et al ., ; Root et al ., ), other work suggests cue‐triggered activity in NAc depends on the subsequent behaviour or outcome of the trial (Nicola et al ., ; Roitman & Loriaux, ). In addition, dorsal striatal and NAc neurons have been previously shown to modulate their activity according to the rat's actions in previous trials (Kim et al ., ; Oyama et al ., ). Consistent with this, here, we found that cue‐related excitatory activity was increased after previous errors (Fig.…”

Section: Discussionmentioning

confidence: 97%

Synchronization in the prefrontal–striatal circuit tracks behavioural choice in a go–no‐go task in rats

Stubbendorff

Molano-Mazón

Young

et al. 2018

Eur J of Neuroscience

View full text Add to dashboard Cite

Rodent striatum is involved in sensory-motor transformations and reward-related learning. Lesion studies suggest dorsolateral striatum, dorsomedial striatum and nucleus accumbens underlie stimulus-response transformations, goal-directed behaviour and reward expectation, respectively. In addition, prefrontal inputs likely control these functions. Here, we set out to study how reward-driven behaviour is mediated by the coordinated activity of these structures in the intact brain. We implemented a discrimination task requiring rats to either respond or suppress responding on a lever after the presentation of auditory cues in order to obtain rewards. Single unit activity in the striatal subregions and pre-limbic cortex was recorded using tetrode arrays. Striatal units showed strong onset responses to auditory cues paired with an opportunity to obtain reward. Cue-onset responses in both striatum and cortex were significantly modulated by previous errors suggesting a role of these structures in maintaining appropriate motivation or action selection during ongoing behaviour. Furthermore, failure to respond to the reward-paired tones was associated with higher pre-trial coherence among striatal subregions and between cortex and striatum suggesting a task-negative corticostriatal network whose activity may be suppressed to enable processing of reward-predictive cues. Our findings highlight that coordinated activity in a distributed network including both pre-limbic cortex and multiple striatal regions underlies reward-related decisions.

show abstract

“…It has been reported that RPE signals can be found in multiple brain areas such as prefrontal cortex, the striatum, GPi, lateral habenula and RMTg (Asaad and Eskandar, 2011; Bromberg-Martin and Hikosaka, 2011; Hong and Hikosaka, 2008; Matsumoto and Hikosaka, 2007, 2008; Oyama et al, 2015). This is potentially because RPE signals are useful for a variety of behaviors outside of reinforcement learning (Den Ouden et al, 2012; Schultz and Dickinson, 2000).…”

Section: Discussionmentioning

confidence: 99%

Distributed and Mixed Information in Monosynaptic Inputs to Dopamine Neurons

Tian

Huang

Cohen

et al. 2016

Neuron

211

233

View full text Add to dashboard Cite

SUMMARY Dopamine neurons encode the difference between actual and predicted reward, or reward prediction error (RPE). Although many models have been proposed to account for this computation, it has been difficult to test these models experimentally. Here we established an awake electrophysiological recording system, combined with rabies virus and optogenetic cell-type identification, to characterize the firing patterns of monosynaptic inputs to dopamine neurons while mice performed classical conditioning tasks. We found that each variable required to compute RPE, including actual and predicted reward, was distributed in input neurons in multiple brain areas. Further, many input neurons across brain areas signaled combinations of these variables. These results demonstrate that even simple arithmetic computations such as RPE are not localized in specific brain areas but rather distributed across multiple nodes in a brain-wide network. Our systematic method to examine both activity and connectivity revealed unexpected redundancy for a simple computation in the brain.

show abstract

Discrete coding of stimulus value, reward expectation, and reward prediction error in the dorsal striatum

Cited by 27 publications

References 48 publications

Striatal and Tegmental Neurons Code Critical Signals for Temporal-Difference Learning of State Value in Domestic Chicks

Striatal and Tegmental Neurons Code Critical Signals for Temporal-Difference Learning of State Value in Domestic Chicks

Synchronization in the prefrontal–striatal circuit tracks behavioural choice in a go–no‐go task in rats

Distributed and Mixed Information in Monosynaptic Inputs to Dopamine Neurons

Contact Info

Product

Resources

About