No abstract

Interval schedules of reinforcement are known to generate habitual behavior, the performance of which is less sensitive to revaluation of the earned reward and to alterations in the action-outcome contingency. Here we report results from experiments using different types of interval schedules of reinforcement in mice to assess the effect of uncertainty, in the time of reward availability, on habit formation. After limited training, lever pressing under fixed interval (FI, low interval uncertainty) or random interval schedules (RI, higher interval uncertainty) was sensitive to devaluation, but with more extended training, performance of animals trained under RI schedules became more habitual, i.e. no longer sensitive to devaluation, whereas performance of those trained under FI schedules remained goal-directed. When the press-reward contingency was reversed by omitting reward after pressing but presenting reward in the absence of pressing, lever pressing in mice previously trained under FI decreased more rapidly than that of mice trained under RI schedules. Further analysis revealed that action-reward contiguity is significantly reduced in lever pressing under RI schedules, whereas action-reward correlation is similar for the different schedules. Thus the extent of goal-directedness could vary as a function of uncertainty about the time of reward availability. We hypothesize that the reduced action-reward contiguity found in behavior generated under high uncertainty is responsible for habit formation.

To understand the neural basis of behavior, it is necessary to record brain activity in freely moving animals. Advances in implantable multi-electrode array technology have enabled researchers to record the activity of neuronal ensembles from multiple brain regions. The full potential of this approach is currently limited by reliance on cable tethers, with bundles of wires connecting the implanted electrodes to the data acquisition system while impeding the natural behavior of the animal. To overcome these limitations, here we introduce a multi-channel wireless headstage system designed for small animals such as rats and mice. A variety of single unit and local field potential signals were recorded from the dorsal striatum and substantia nigra in mice and the ventral striatum and prefrontal cortex simultaneously in rats. This wireless system could be interfaced with commercially available data acquisition systems, and the signals obtained were comparable in quality to those acquired using cable tethers. On account of its small size, light weight, and rechargeable battery, this wireless headstage system is suitable for studying the neural basis of natural behavior, eliminating the need for wires, commutators, and other limitations associated with traditional tethered recording systems.

The timing of actions is critical for adaptive behavior. In this study we measured neural activity in the substantia nigra as mice learned to change their action duration to earn food rewards. We observed dramatic changes in single unit activity during learning: both dopaminergic and GABAergic neurons changed their activity in relation to behavior to reflect the learned instrumental contingency and the action duration. We found the emergence of "action-on" neurons that increased firing for the duration of the lever press and mirrorimage "action-off" neurons that paused at the same time. This pattern is especially common among GABAergic neurons. The activity of many neurons also reflected confidence about the just completed action and the prospect of reward. Being correlated with the relative duration of the completed action, their activity could predict the likelihood of reward collection. Compared with the GABAergic neurons, the activity of dopaminergic neurons was more commonly modulated by the discriminative stimulus signaling the start of each trial, suggesting that their phasic activity reflected sensory salience rather than any reward prediction error found in previous work. In short, these results suggest that (1) nigral activity is highly plastic and modified by the learning of the instrumental contingency; (2) GABAergic output from the substantia nigra can simultaneously inhibit and disinhibit downstream structures, while the dopaminergic output also provide bidirectional modulation of the corticostriatal circuits; (3) dopaminergic and GABAergic neurons show similar task-related activity, although DA neurons are more responsive to the trial start signal.