Dynamics of time matching: Arousal makes better seem worse

Abstract

Matching of time allocation across alternatives in proportion to relative reinforcement rates is a ubiquitous finding in the animal-learning literature on choice. The dynamics ofthe underlying mechanism, however, remain poorly understood. A recent finding by Belke (1992) profoundly challenges scalar expectancy theory (SET; Gibbon et al., 1988) and other accounts of matching in concurrent variable interval (VI) schedules. He studied concurrent probe tests of stimuli associated with equal VIs but trained in alternative concurrent pairs. In training, one was preferred and the other not. Unreinforced probes revealed a strong preference for the alternative preferred in training. An experiment is reported replicating this result and showing that it is not due to generalization of preference levels from training. When the probe is between the two preferred training stimuli, the richer schedule is unpreferred. A SET account of these results is presented which implicates two processes in time allocation: (1) the choice between alternatives based on memory for delays to reinforcement, and (2) the times at which such choices are made. The former process is sensitive to reinforcement scheduling; the latter is sensitive to arousal levels induced by overall reinforcement rates in training.A now venerable tradition in operant schedule research shows that animals choosing between concurrently available variable schedules of reward allocate time spent at each alternative in proportion to their rates ofpayoff(the matching law; Herrnstein, 1970; see Williams, 1988, for a review). The matching law is ubiquitous throughout the animal kingdom. A form of matching is seen at the level of groups of animals distributing individuals across resource sites where it is known as the "ideal free distribution" (see Gallistel, 1990, for a review). At the level of individuals, where it has been studied extensively in the animal-learning literature, accounts of matching have engendered an enduring controversy about its source. Matching of behavior proportionate to payoff implies more choices for the less profitable resource than strictly necessary to maximize overall payoff rates. Several "molecular" or "quasi-molecular" accounts of matching, which posit maximizing payoffs at a local or molecular level, have been proposed. For example, meloriation (Herrnstein & Vaughan, 1980;Vaughan, 1985) and "momentary maximizing" (Shimp, 1969) argue that subjects choose the better of two local rates or probabilities of reinforcement, and this results in overall matching. Rather than explaining it, other, "molar" matching accounts simply assume matching (e.g., Baum & Rachlin, 1969; Gallistel, 1990;Mark & Gallistel, 1994).The author is grateful for extensive discussion of these ideas with C. R. Gallistel, who lent formative insights into an understanding of switching rates. Thanks are due S. Fairhurst for data collection and analysis. The work was supported by NIMH Grant MH41649-09. Address correspondence to John Gibbon, Biopsychology Unit 50, 722 W. I68th St.,...

Context-dependent utility overrides absolute memory as a determinant of choice

“…Here, we test the idea with an experimental design that pitches absolute against comparative processes of choice (11). We use European starlings (Sturnus vulgaris) as subjects and manipulate the utility of different options by altering their relative ranking through context manipulations, as in earlier experiments with pigeons (12)(13)(14). The starlings pecked at colored response keys (options) to obtain food rewards.…”

A Test of Gibbon's Feedforward Model of Matching

“…This important fact was first reported by Heyman (1982) and later confirmed by Gibbon (1995). It implies that the termination of a visit to a location where rewards are found is determined by a process analogous to the repeated flipping of a coin.…”

“…

Gibbon (1995) elaborated an ingenious model of matching, a feedforward model that is consistent with Heyman's (1982) suggestion that matching behavior does not depend on selection by consequences. Most models (for example, Herrnstein & Vaughan, 1980) have been feedback models, built on the law of effect.

…”

“…John Gibbon pioneered the psychophysical study of interval timing and the application of information-processing models to our understanding of conditioned behavior. Among his many, highly original contributions was a model of matching behavior (Gibbon, 1995), which differed in a fundamental way from previous models. The difference has potentially far reaching implications for our understanding of instrumentally conditioned behavior.…”

The Neural Mechanisms that Underlie Decision Making

“…They are not making the choice they appear to be making. Rather, they are deciding when to abandon whichever option they have already chosen ( Belke, 1992 ;Gibbon, 1995 ;Gallistel and Gibbon, 2000 ;Gallistel et al ., 2001 ).…”