Choice, delay, probability, and conditioned reinforcement

Abstract

The hyperbolic-decay model is a mathematical expression of the relation between delay and reinforcer value. The model has been used to predict choices in discrete-trial experiments on delay-amount tradeoffs, on preference for variable over fixed delays, and on probabilistic reinforcement. Experiments manipulating the presence or absence of conditioned reinforcers on trials that end without primary reinforcement have provided evidence that the hyperbolic-decay model actually predicts the strength of conditioned reinforcers rather than the strength of delayed primary reinforcers. The model states that the strength of a conditioned reinforcer is inversely related to the time spent in its presence before a primary reinforcer is delivered. A possible way to integrate the model with Grace's (1994) contextual-choice model for concurrent-chain schedules is presented. Also discussed are unresolved difficulties in determining exactly when a stimulus will or will not serve as a conditioned reinforcer. 131One of the most potent factors affecting an individual's choice behavior is delay of reinforcement. With animal subjects, numerous studies have shown that delays of just a few seconds can have dramatic affects on choice behavior (e.g., Ainslie, 1974; Green, Fisher, Perlow, & Sherman, 1981;Rachlin & Green, 1972). Human subjects can sometimes tolerate long delays, but there have been many examples in which briefdelays have affected human choice behavior as well (e.g., Millar & Navarick, 1984;Navarick, 1985Navarick, , 1986 Solnick, Kannenberg, Eckerman, & Waller, 1980). Besides being powerful, the effects of reinforcer delay are also ubiquitous: It is difficult to think of many choice situations, either inside or outside the laboratory, that do not involve delays in one way or another. Consider, for example, two of the most common research procedures in operant research on choice-i-concurrent schedules and concurrent-chain schedules. By definition, concurrent schedules involve a choice between two or more simultaneously available reinforcement schedules. Except in the special case of continuous reinforcement schedules (CRF), in which every response is followed by a reinforcer, two concurrent schedules will each include delays (determined by the schedule, the subject's behavior, or both) between the first response on a schedule and the eventual delivery ofa reinforcer. In concurrent-chain schedules, the delays are even more numerous, because these procedures Preparation of this manuscript was supported by Grant MH 38357 from the National Institutes of Menta! Health, I thank Michael Davison and Randy Grace for providing me with data sets from their research. Correspondence should be addressed to 1. E. Mazur. Psychology Department. Southern Connecticut State University. New Haven. CT 06515 (e-mail: mazur@scsu.ctstateu.edu).Note: This article is one of those occasionally invited by the editor in which authors have the opportunity to provide an overview of their research programs.-Editor include both "initial links" and "termina...

Operant Conditioning

“…Many experimental and theoretical papers on conditioned reinforcement in pigeons and rats have been published since the early 1960s using some version of the concurrent chains procedure of Autor (1960Autor ( , 1969. These studies have demonstrated a number of functional relations between rate measures and have led to several closely related theoretical proposals such as a version of the matching law, incentive theory, delay-reduction theory, and hyperbolic value-addition (e.g., Fantino 1969a,b;Grace 1994;Herrnstein 1964;Killeen 1982;Killeen & Fantino 1990;Mazur 1997Mazur , 2001Williams 1988Williams , 1994Williams , 1997. Nevertheless, there is as yet no theoretical consensus on how best to describe choice between sources of conditioned reinforcement, and no one has proposed an integrated theoretical account of simple chain and concurrent chain schedules.…”

Basolateral amygdala lesions and sensitivity to reinforcer magnitude in concurrent chains schedules

“…Both the left and right stimulus lights shut off at the onset of the terminal link, which was segmented by tones in the first and last 2 s of terminal links greater than FT 6 s; a tone occurred in the first 2 s of FT 2-and 4-s terminal links. The tone segmented the delay because stimuli of longer duration may be less efficacious conditioned reinforcers [36]. The frequency of the tone (1 or 15 kHz, 75 dB) signaled the reinforcer magnitude.…”

The behavioral economics of will in recovery from addiction

“…If future choices of smaller-sooner vs. larger-later alternatives were bundled with present choices--e.g., if the choice of whether to get drunk now at the expense of feeling good tomorrow entailed committing to the same alternative for every day for the next week--the total value of the larger-later bundle relative to the smaller sooner bundle would increase, since the discounted value of the larger-later alternative in each future choice is greater than that of the smaller-sooner. Assuming that the discounted value of series of rewards is additive (as appears to be the case; Mazur, 1997), then bundling future identical choices with a current choice can reduce impulsive choice, given hyperbolic delay discounting but not exponential delay discounting (Figure 2a and 2b). 1 Experiments with both human and rodent subjects confirm a greater tolerance for delay with bundled rewards.…”