Conditioned response timing and integration in the cerebellum.

Abstract

Classical conditioning procedures instill knowledge about the temporal relationships between events. The unconditioned stimulus (US) is the event to be timed. The conditioned response (CR) is viewed as a prediction of the inlminence of the US. Knowledge of the elapsed time between conditioned stimuli (CSs) and US delivery is expressed in the topological features of the CR. The peak amplitude of the CR coincides with the timing of the US. A simple connectionist network based on Sutton and Barto's Time Derivative (TD) Model of Pavlovian Reinforcement provides a mechanism that can account for and simulate CR timing in a variety of protocols. This article describes extensions of the model to predictive timing under temporal uncertainty. The model is expressed in terms of equations that operate in real time according to a competitive learning rule.

Backward inhibitory learning in honeybees: a behavioral analysis of reinforcement processing.

“…One major approach for understanding the neural basis of behavioral learning, which has recently been applied to several learning paradigms, is to investigate how real-time computational models might be implemented by neural circuits that are involved in learning. For both the SOP model and the TD model, implementations within the cerebellar circuit that support motor learning have been suggested [SOP (Wagner and Donegan 1989); TD (Moore and Choi 1997)]. The TD model, moreover, directly predicts features of the mammalian reward processing system (Barto 1995;Houk et al 1995;Montague et al 1996;Schultz et al 1997), and a simplified version of TD has been suggested to predict properties of the reward processing VUMmxl neuron in the honeybee, as well as predictive learning during foraging (Montague et al 1995;see also Hammer 1997).…”

“…In particular, Sutton and Barto's (1990) time-derivative (TD) model is able to predict the ISI dependence of both excitatory and inhibitory learning (Malaka and Hammer 1996). Moreover, the TD model has been used recently in several variants to describe the properties of neural systems that process reinforcement and to propose putative neural implementations of predictive learning (Barto 1995;Houk et al 1995;Montague et al 1995Montague et al , 1996Moore and Choi 1997;Schultz et al 1997; for a similar approach on value processing, see also Friston et al 1994). In the tradition of the Rescorla and Wagner (1972) model, the TD algorithm makes inhibitory learning dependent on the presence of context stimuli with positive associative strength (Malaka and Hammer 1996).…”

Blockade of GABAA receptors in the interpositus nucleus modulates expression of conditioned excitation but not conditioned inhibition of the eyeblink response

“…It is possible that the parallel fiber-alone stimulation that presumably occurs during the non-reinforced compound stimulus in conditioned inhibition training induces LTP in Purkinje cells. LTP would amplify Purkinje cell inhibition of the deep nuclei when the inhibitory stimulus was presented and in turn inhibit the production CRs (Freeman & Nicholson, 1999;Mauk & Donegan, 1997;Moore & Choi, 1997). As mentioned above, several models argue that excitatory conditioning involves LTD of parallel fiber inputs to Purkinje cells (e.g., Thompson & Krupa, 1994).…”

“…The mechanism underlying the hypothesized increased Purkinje cell inhibition during conditioned inhibition could be long-term potentiation (LTP) of the parallel fiber inputs to Purkinje cells (Choi, 1999;Freeman & Nicholson, 1999;Mauk & Donegan, 1997;Moore & Choi, 1997). Paired stimulation of parallel and climbing fibers induces LTD (Chen & Thompson, 1995;Schreurs, Oh, & Alkon, 1996;Freeman, Shi, & Schreurs, 1998), but parallel fiber stimulation without climbing fiber stimulation can induce LTP (Sakurai, 1987;Salin, Malenka, & Nicoll, 1996).…”

Timing in trace conditioning of the nictitating membrane response of the rabbit (Oryctolagus cuniculus): Scalar, nonscalar, and adaptive features

“…The theoretical aim of these studies was to inform models of NM conditioning, which assume that CR timing is mediated by a spectrum of microstimuli initiated by the conditioned stimulus (CS) (Desmond and Moore 1988;Grossberg and Schmajuk 1989;Gluck et al 1990;Sutton and Barto 1990;Buonomano and Mauk 1994;Machado 1997;Kirkpatrick and Church 1998;Buhusi and Schmajuk 1999;Vogel et al 2003;Ludvig et al 2008Ludvig et al , 2009. Their neural counterpart resides in activation across the cerebellar cortex (Buonomano and Mauk 1991;Moore and Choi 1997;Mauk et al 2000), including the planar arrangement of Purkinje cells and their dendritic morphology (Ito 1984;Steuber and Willshaw 2004). In fact, interstimulus interval (ISI)-dependent activity has appeared in Purkinje cells (Jirenhed et al 2007).…”