Tonically active neurons in the striatum differentiate between delivery and omission of expected reward in a probabilistic task context

Abstract

Tonically active neurons (TANs) in the primate striatum are responsive to rewarding stimuli and they are thought to be involved in the storage of stimulus-reward associations or habits. However, it is unclear whether these neurons may signal the difference between the prediction of reward and its actual outcome as a possible neuronal correlate of reward prediction errors at the striatal level. To address this question, we studied the activity of TANs from three monkeys trained in a classical conditioning task in which a liquid reward was preceded by a visual stimulus and reward probability was systematically varied between blocks of trials. The monkeys' ability to discriminate the conditions according to probability was assessed by monitoring their mouth movements during the stimulus-reward interval. We found that the typical TAN pause responses to the delivery of reward were markedly enhanced as the probability of reward decreased, whereas responses to the predictive stimulus were somewhat stronger for high reward probability. In addition, TAN responses to the omission of reward consisted of either decreases or increases in activity that became stronger with increasing reward probability. It therefore appears that one group of neurons differentially responded to reward delivery and reward omission with changes in activity into opposite directions, while another group responded in the same direction. These data indicate that only a subset of TANs could detect the extent to which reward occurs differently than predicted, thus contributing to the encoding of positive and negative reward prediction errors that is relevant to reinforcement learning.

The Role of Striatal Tonically Active Neurons in Reward Prediction Error Signaling during Instrumental Task Performance

“…In our previous study, we showed that some TANs responded to reward and no reward with decreases and increases in firing, respectively (Apicella et al, 2009). These opposite response patterns may reflect positive and negative errors in prediction of reward, consistent with the rules of temporal difference models of associative learning (Schultz et al, 1997;Dayan and Balleine, 2002;Montague and Berns, 2002).…”

“…For each neuron, a normalized perievent time histogram was obtained by dividing the content of each bin by the number of trials, and the population histogram was constructed by averaging all normalized histograms. We also used a time-window analysis to statistically assess and compare changes in the average population response between conditions (Apicella et al, 2009). First, latency and duration of these changes were determined for each population histogram.…”

“…The two monkeys used in the present experiment were also trained in a classical conditioning task [monkeys 2 and 3 in Apicella et al (2009)]. The activity of 59 TANs (31 in monkey P and 28 in monkey G) was examined during performance of the classical conditioning task while varying reward probability in a similar way, using the same block design.…”

“…However, it remains unclear whether striatal neurons encode this type of prediction error (Schultz and Dickinson, 2000;Niv and Schoenbaum, 2008;Roesch et al, 2010), although it was recently reported that the activity of a group of striatal neurons in rats, presumed projection neurons, was linked to discrepancies between outcomes and their predictions, which suggests a coding of prediction error (Oyama et al, 2010). Singleneuron recording studies in monkeys have also implicated the changes in activity of a particular class of striatal neurons, termed tonically active neurons (TANs) and thought to be cholinergic interneurons, as a possible neuronal substrate of the prediction error signal (Joshua et al, 2008;Apicella et al, 2009). However, as in the study by Oyama et al (2010), these findings were reported in Pavlovian (i.e., classical) conditioning in which no behavioral response is required for reward, whereas no evidence of prediction error signaling has been observed during an instrumental task in which the rewarding outcome depends on an action (Morris et al, 2004).…”

Reward Prediction Error Coding in Dorsal Striatal Neurons

“…However, on the neuron level, there have recently been contradictory reports concerning the RPE coding in the striatum. Morris et al (2004) recorded the activity of TANs (tonically active neurons), or striatal cholinergic interneurons, in monkeys and concluded that TANs in monkeys do not code RPE, whereas Apicella et al (2009) have reported that they responded to RPE in a fashion inverse to that of dopamine neurons, that is, they show inhibitory activity for a positive RPE. Roesch et al (2009) recorded the activity of rat striatal neurons that were apparently medium spiny projection neurons and reported that only few of them appeared to code RPE signals, whereas Kim et al (2009) found both inverse and noninverse RPE coding in several medium spiny neurons and interneurons.…”