Classical conditioning as a nonstationary, multivariate time series analysis: A spreadsheet model

Abstract

The implementation of the Gallistel (1990) model of classical conditioning on a spreadsheet with matrix operations is described. The model estimates the Poisson rate of unconditioned stimulus (US) occurrence in the presence of each conditioned stimulus (eS). The computations embody three implicit principles: additivity (of the rates predicted by each eS), provisional stationarity (the rate predicted by a given es has been constant over all the intervals when that es was present), and predictor minimization (when more than one solution is possible, the model minimizes the number of ess with a nonzero effect on US rate). The Kolmogorov-Smirnov statistic is used to test for nonstationarity. There are no free parameters in the learning model itself and only two parameters in the formally specified decision process, which translates what has been learned into conditioned responding. The model predicts a wide range of conditioning phenomena, notably: blocking, overshadowing, overprediction, predictive sufficiency, inhibitory conditioning, latent inhibition, the invariance in the rate of conditioning under scalar transformation of es-us and US-US intervals, and the effects of partial reinforcement on acquisition and extinction.Gallistel (1990, in press) describes a model of the classical conditioning process in which it is assumed that what the animal learns is the rate of unconditioned stimulus (US) occurrence to be expected in the presence of a conditioned stimulus (CS). The model predicts many experimental results that have posed problems for past or present associative models of conditioning, including: (1) the effect of partial reinforcement on the rate of acquisition and the rate of extinction (Gibbon, Farrell, Locurto, Duncan, & Terrace, 1980); (2) the effect of the duty cycle (or ITIIISI ratio) on the rate of acquisition and its lack of effect on the rate of extinction (Gibbon, Baldock, Locurto, Gold, & Terrace, 1977); (3) blocking and overshadowing (Kamin, 1969); (4) the blocking effect of background conditioning (Rescorla, 1968); (5) the effects of having the "background" USs signaled by another CS (Robbins & Rescorla, 1989); (6) inhibitory conditioning when the CS and US are explicitly unpaired; (7) the predictive sufficiency results of Wagner, Logan, Haberlandt, and Price (1968), in which the CS that accounts for more of the variance in US occurrence is the CS that gets conditioned; (8) inhibitory conditioning in overpredictionexperiments (Kremer, 1978); and (9) the noninhibitory retarding effect of a "latent inhibition" training phase on the rate of subsequent conditioning (Reiss & Wagner, 1972).I am grateful to Tom Wickens for discussions that led to the algorithm for testing stationarity. The costs of creating this spreadsheet and preparing the manuscript were partially covered by NSF Grant BNS89-96246. Correspondence should be addressed to the author at the Department of Psychology, 405 Hilgard Ave., University of California, Los Angeles, CA 90024-1563.The model assumes that USs (brief shocks or small...

Time, rate, and conditioning.

“…From a timing perspective, acquisition is a consequence of decisions that the animal makes about whether to respond to a CS. Our models for these decisions are adapted from Gallistel's (1990Gallistel's ( , 1992aGallistel's ( , 1992b earlier accounts, which we call RET. In our acquisition model, the decision to respond to the CS in the course of conditioning is based on the animal's growing certainty that the CS has a substantial effect on the rate of reinforcement.…”

Time and Associative Learning.

“…The upper limit is imposed by the fact that the sum of the rates ascribed to two or more predictors must equal the rates observed during periods when predictors are simultaneously present. A spreadsheet implementation of RET (Gallistel, 1992) is available from CRG. Readers may use it to verify that it does predict the Durlach (1983) result.…”

Extinction from a rationalist perspective

“…From a rationalist perspective, learning is mediated by problem-specific computational mechanisms (Gallistel 1990; 1992; 1995). The theorist's first and most important challenge is to correctly characterize the problems that the mechanisms informing the animal's behavior have evolved to solve.…”