Concurrent random interval schedules of reinforcement

Killeen, Peter R.; Shumway, Gary S.

doi:10.3758/bf03332482

Cited by 4 publications

(3 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We are not the first to superimpose hold contingencies on operant reinforcement schedules, for example, as in limited hold procedures (see Boelens, 1984; Buskist & Morgan, 1987; Morse, 1966). As far as we know, however, we are the first to apply hold equally for all choices and to test whether choice distributions—under conditions spanning the range from probability learning to concurrent reinforcement—can be explained by differences in hold (see Killeen & Shumway, 1971, for related research).…”

mentioning

confidence: 99%

Choice as a function of reinforcer "hold": From probability learning to concurrent reinforcement.

Jensen

Neuringer

2008

Journal of Experimental Psychology: Animal Behavior Processes

View full text Add to dashboard Cite

Two procedures commonly used to study choice are concurrent reinforcement and probability learning. Under concurrent-reinforcement procedures, once a reinforcer is scheduled, it remains available indefinitely until collected. Therefore reinforcement becomes increasingly likely with passage of time or responses on other operanda. Under probability learning, reinforcer probabilities are constant and independent of passage of time or responses. Therefore a particular reinforcer is gained or not, on the basis of a single response, and potential reinforcers are not retained, as when betting at a roulette wheel. In the "real" world, continued availability of reinforcers often lies between these two extremes, with potential reinforcers being lost owing to competition, maturation, decay, and random scatter. The authors parametrically manipulated the likelihood of continued reinforcer availability, defined as hold, and examined the effects on pigeons' choices. Choices varied as power functions of obtained reinforcers under all values of hold. Stochastic models provided generally good descriptions of choice emissions with deviations from stochasticity systematically related to hold. Thus, a single set of principles accounted for choices across hold values that represent a wide range of real-world conditions. Keywords matching; stochastic response; limited hold; reinforcement probability; pigeons Probability learning and concurrent reinforcement are procedures commonly used to study choice. They share many similarities. For example, subjects choose repeatedly between two options to obtain reinforcers: Human participants press buttons or push computer keys for points or money; monkeys look left or right for orange juice or raisins; rats run down alleys or press levers for food pellets; and pigeons peck keys for grain. Choice distributions under both procedures are influenced by reinforcer distributions and by other attributes of the reinforcers, for example, qualities, amounts, and delays. However, aspects of the procedures differ, and there is controversy concerning results, especially whether results from the two procedures are consistent.Under probability-learning procedures, reinforcer availability depends on a random number generator that is activated (or fired) each time a choice occurs. Thus each choice is reinforced with some probability, and the probabilities are independent of previous events. For example, a left (L) choice might be reinforced with 0.4 probability and a right (R) one with 0.1 probability. In some studies the resulting response distributions are described as probability matching: Ratios of responses (L/R) equal ratios of reinforcement probabilities, or 4 to 1 in the example given (Myers, Lohmeier, & Well, 1994;Vulkan, 2000). Such matching is inefficient:Correspondence concerning this article should be addressed to Allen Neuringer, Psychology Department, Reed College, Portland, OR 97202. E-mail: E-mail: allen.neuringer@reed.edu. NIH Public Access NIH-PA Author ManuscriptNIH-PA Author Manusc...

show abstract

mentioning

confidence: 99%

Choice as a function of reinforcer "hold": From probability learning to concurrent reinforcement.

Jensen

Neuringer

2008

Journal of Experimental Psychology: Animal Behavior Processes

View full text Add to dashboard Cite

show abstract

“…The longer you respond on one option, the more likely it is that reinforcement has set up on the other (MacDonall, 2005). As the hold is shortened, behavior shifts from matching to the nominal rates, to maximizing (Killeen & Shumway, 1971), driven by the positive feedback loop as animals approach ever greener pastures.…”

Section: Classic Schedulesmentioning

confidence: 99%

Theory of reinforcement schedules

Killeen

2023

J Exper Analysis Behavior

View full text Add to dashboard Cite

The three principles of reinforcement are (1) events such as incentives and reinforcers increase the activity of an organism; (2) that activity is bounded by competition from other responses; and (3) animals approach incentives and their signs, guided by their temporal and physical conditions, together called the “contingencies of reinforcement.” Mathematical models of each of these principles comprised mathematical principles of reinforcement (MPR; Killeen, 1994). Over the ensuing decades, MPR was extended to new experimental contexts. This article reviews the basic theory and its extensions to satiation, warm‐up, extinction, sign tracking, pausing, and sequential control in progressive‐ratio and multiple schedules. In the latter cases, a single equation balancing target and competing responses governs behavioral contrast and behavioral momentum. Momentum is intrinsic in the fundamental equations, as behavior unspools more slowly from highly aroused responses conditioned by higher rates of incitement than it does from responses from leaner contexts. Habits are responses that have accrued substantial behavioral momentum. Operant responses, being predictors of reinforcement, are approached by making them: The sight and feel of a paw on a lever is approached by placing paw on lever, as attempted for any sign of reinforcement. Behavior in concurrent schedules is governed by approach to momentarily richer patches (melioration). Applications of MPR in behavioral pharmacology and delay discounting are noted.

show abstract

“…Without constant-probability VI (viz., RI) schedules, allocation becomes inhomogeneous in time (Baum, 1979). Without the unlimited hold in typical schedules (Jensen & Neuringer, 2008;Killeen & Shumway, 1971) allocation deviates from perfect matching. In concurrent variable ratio (VR) schedules, response allocation favors the richest, and that often drives reinforcement and behavior to exclusivity.…”

mentioning

confidence: 99%

The logistics of choice

Killeen

2015

J Exper Analysis Behavior

View full text Add to dashboard Cite

The generalized matching law (GML) is reconstructed as a logistic regression equation that privileges no particular value of the sensitivity parameter, a. That value will often approach 1 due to the feedback that drives switching that is intrinsic to most concurrent schedules. A model of that feedback reproduced some features of concurrent data. The GML is a law only in the strained sense that any equation that maps data is a law. The machine under the hood of matching is in all likelihood the very law that was displaced by the Matching Law. It is now time to return the Law of Effect to centrality in our science.

show abstract

Concurrent random interval schedules of reinforcement

Cited by 4 publications

References 4 publications

Choice as a function of reinforcer "hold": From probability learning to concurrent reinforcement.

Choice as a function of reinforcer "hold": From probability learning to concurrent reinforcement.

Theory of reinforcement schedules

The logistics of choice

Contact Info

Product

Resources

About