Habit Learning by Naive Macaques Is Marked by Response Sharpening of Striatal Neurons Representing the Cost and Outcome of Acquired Action Sequences

Abstract

SUMMARY Over a century of scientific work has focused on defining the factors motivating behavioral learning. Observations in animals and humans trained on a wide range of tasks support reinforcement learning (RL) algorithms as accounting for the learning. Still unknown, however, are the signals that drive learning in naïve, untrained subjects. Here, we capitalized on a sequential saccade task in which macaque monkeys acquired repetitive scanning sequences without instruction. We found that spike activity in the caudate nucleus after each trial corresponded to an integrated cost-benefit signal that was highly correlated with the degree of naturalistic untutored learning by the monkeys. Across learning, neurons encoding both cost and outcome gradually acquired increasingly sharp phasic trial-end responses that paralleled the development of the habit-like, repetitive saccade sequences. Our findings demonstrate a novel integrated cost-benefit signal by which RL and its neural correlates could drive naturalistic behaviors in freely behaving primates.

Pallidal spiking activity reflects learning dynamics and predicts performance

“…Along with the STN, the striatum is the main input structure to the GPe. Studies examining striatal activity in cognitively demanding tasks did not show any modulation in the activity during the time epoch preceding the choice of the animal as choices became more and more habitual and automated (7,29,39). However, there is consensus that the value associated with an outcome, its cost, and their interaction are all represented in striatal neuronal activity (7).…”

“…Dopamine modifies the efficacy of the corticostriatal synapses, as well as the striatal excitability directly, using signals that incorporate both expectation and external gains and losses (3)(4)(5). Decades of research have revealed the manner in which signals relaying information concerning expected and actual gains and costs are incorporated in the striatal dynamic system (6)(7)(8)(9). However, these dopamine-and striato-centric views often fail to take into account our current understanding of the basal ganglia, which acknowledges that the actor (main axis) network has more than one input/output hub and employs multiple reciprocally and feed-forward connected computational components (10).…”

Bursts of beta oscillation differentiate postperformance activity in the striatum and motor cortex of monkeys performing movement tasks

“…Numerous studies in nonhuman primates have demonstrated significant task-related effects in the low beta band (4,5,(27)(28)(29), but except for the elegant studies of Tan and colleagues (30,31), oscillation during the postmovement period has not been analyzed in detail. Brief episodes of heightened spike activity at task end have been documented in both nonhuman primates and rodents (16)(17)(18)(19)(20)32). It has been suggested that the end-signaling could be part of a mechanism to tag completion of successful sequences of behavior, providing action boundaries to facilitate the expression of such action sequences (16,17).…”

Translational Research on Habit and Alcohol

“…One straightforward way to promote habitual behavior is to overtrain animals – that is, prolonged training such that the operant response becomes automatic. Overtraining is often used to study habitual behavior in non-human primates [45–49] and used less often in rodents [1, 14]. Monkeys are trained to execute automatic actions through a long-term learning period including thousands of action-outcome parings [for example, see 50].…”

“…Further, electrophysiological studies have found distinct firing patterns of neurons in the dorsolateral striatum during habit-like reward seeking as opposed to goal-directed seeking [53, 54]. The primate homolog of the rodent dorsolateral striatum, the putamen, plays a similar critical role in habitual or automatic behavior [45, 47–49]. Moreover, numerous studies have indicated involvement of neocortical and allocortical regions in the establishment and performance of habitual actions.…”