Addendum to Killeen's (2019) Bidding for Delayed Rewards

Self Cite

The three principles of reinforcement are (1) events such as incentives and reinforcers increase the activity of an organism; (2) that activity is bounded by competition from other responses; and (3) animals approach incentives and their signs, guided by their temporal and physical conditions, together called the “contingencies of reinforcement.” Mathematical models of each of these principles comprised mathematical principles of reinforcement (MPR; Killeen, 1994). Over the ensuing decades, MPR was extended to new experimental contexts. This article reviews the basic theory and its extensions to satiation, warm‐up, extinction, sign tracking, pausing, and sequential control in progressive‐ratio and multiple schedules. In the latter cases, a single equation balancing target and competing responses governs behavioral contrast and behavioral momentum. Momentum is intrinsic in the fundamental equations, as behavior unspools more slowly from highly aroused responses conditioned by higher rates of incitement than it does from responses from leaner contexts. Habits are responses that have accrued substantial behavioral momentum. Operant responses, being predictors of reinforcement, are approached by making them: The sight and feel of a paw on a lever is approached by placing paw on lever, as attempted for any sign of reinforcement. Behavior in concurrent schedules is governed by approach to momentarily richer patches (melioration). Applications of MPR in behavioral pharmacology and delay discounting are noted.

Section: Classic Schedulesmentioning

confidence: 71%

Theory of reinforcement schedules

Killeen

2023

Self Cite

“…This is not the first proposal to rectify the hyperbola. Killeen (2019Killeen ( , 2020 inverted Equation 1 and fit linear regressions to the results, treating them as predicting the amount by which a remote good would need to be incremented to generate parity with the more proximal one. Berk et al (2021) inverted Equation 2and rearranged it so that the ratio of the value lost due to its delay to the value preserved despite its delay, which they labeled the "immediacy premium," was on one side.…”

Section: Discussion and Summarymentioning

confidence: 99%

From data through discount rates to the area under the curve

Killeen

2023

Self Cite

The rate of discounting future goods is a crucial factor in intertemporal trade‐offs, upon which depends not only individual well‐being but also that of our planet: How much privation now for a temperate future for our grandchildren? What is the best way to measure how the value of future goods decreases with its delay? The most accurate discount functions involve several covarying parameters, making interpretation equivocal. A universal and robust measure is the area under the discount curve, the AuC. The AuC of a hyperbolic discount function is a logarithmic function of the discount rate, k. The same integral also approximates the area under a hyperboloid function. A simple technique converts each datum into estimates of the discount rate, eliminating rogue data points in the process. These trimmed estimates are converted into areas and tested against data, where they succeed at predicting the AuC and its relation to log(k).

“…Replace r in Equation 2 with 1/ d and simplify, and it comes to look very much like Equation 3 (see Killeen, 2020, p. 685). De Villiers and Herrnstein (1976) used such a reinterpreted version of Equation 2 to successfully predict the effect of delayed reinforcement on response latency, speed, and response rate in five published experiments.…”

Section: Mazur's Hyperbolamentioning

confidence: 99%

“…To test this, I increased the cost of moving to the hopper by shifting it up to 8 ft (2.44 m) away from the lever. The rats increased their accumulation of pellets as a linear function of the distance (Killeen, 2019b, 2020; Equation 12 below). Eventually, another light went on.…”

Section: Linearization Of the Functionsmentioning

confidence: 99%

Hyperbolae

Killeen

2024

Self Cite

Hyperbolic relations between independent and dependent variables are ubiquitous in the experimental analysis of behavior, mentioned in over 150 articles in the Journal of the Experimental Analysis of Behavior. There are two principal forms of hyperbolae: The first describes the relation between response rate and reinforcement rate on variable‐interval schedules of reinforcement; it rises asymptotically toward a maximum. The second describes the relation between the current equivalent value of an incentive and its delay or (im)probability; it falls from a maximum toward an asymptote of 0. Where do these come from? What do their parameters mean? How are they related? This article answers the first two questions and addresses the last.