Integration of reinforcement effects over time

Abstract

Two accounts of how density of reinforcement affects steady-state performance on probabilistic schedules were compared: the real-time linear operator (RTLO) model and a temporal control model (in which response strength is determined by reinforcement probability as a function of postreinforcement time). In Experiment 1, the probability of reinforcement repeatedly cycled between extinction and a random-ratio 10 schedule. Response-rate gain and phase did not change with period of the cycle as predicted by the RTLO model, nor did either model predict the differences in response rate following reinforcement at different points in the cycle. In Experiment 2, the probability of reinforcement was elevated immediately following a reinforcement but fell after a few seconds. Previous reinforcements had no effect upon responding. An extension of the temporal control model, the cumulative impulse model, allowed for the summing of response strength over successive reinforcements and was consistent with the data of both experiments.Many accounts of foraging and behavior on reinforcement schedules (Charnov, 1976;Herrnstein, 1961Herrnstein, , 1990Herrnstein & Vaughan, 1980) assume that behavior is driven by the animal's estimate of prey density or foodreinforcement rate. How do animals form this estimate from the stream of discrete feeding episodes? Two very different approaches have been taken to this problem. The most popular view in the literature on foraging is that behavior is guided by some form of moving average of local reinforcement rate, referred throughout here as the real-time linear operator (RTLO) model, described by Lea and Dow (

Time and Memory: Towards a Pacemaker‐free Theory of Interval Timing

“…Events preceding the most recent time marker affect V M ; hence, they affect the animal's ''time estimate.'' There are similar effects on reinforcement schedules: For example, the postfood response profile shows proactive interference effects on some temporal schedules (e.g., Lejeune, Ferrara, Simons, & Wearden, 1997;Staddon, 1974aStaddon, , 1974b, and the effects of successive reinforcers have been shown to cumulate under some conditions (Horner, Staddon, & Lozano, 1997). The fact that events have cumulative effects in the MTS model automatically subsumes timing and counting under the same theory, as we show in a moment.…”

Linear-Nonlinear-Poisson Models of Primate Choice Dynamics

“…Linear techniques have been employed, for example, in studying the dynamics of extinction (Palya, Walter, Kessel, & Lucke, 1996, session-to-session changes in behavior under concurrent variable-interval (VI) reward schedules (Hunter & Davison, 1985), integration of reward effects over time (Horner, Staddon, & Lozano, 1997), and to establish a theoretical basis for the steady-state matching relation first enunciated by Herrnstein (McDowell, 1980;McDowell, Bass, & Kessel, 1983;McDowell & Kessell, 1979). Our use of the more general LNP framework both extends these methods and applies them in a new behavioral context.…”

Wiener Filter Estimation of Transfer Functions

“…Behaviorally, such resonance spikes would correspond to strong 22-s period oscillation in the response rate, a finding that is not seen in the data. More broadly, Palya et al's (2002) results and those of other workers studying behavioral dynamics on a fine scale (Higa, 1996;Higa & Pierson, 1998;Horner, Staddon & Lozano, 1997) indicate that sharp transitions in reinforcement are not reflected in changes in behavior that occur faster than the variations of a 30 mHz frequency. This implies that the transfer function corresponds to some form of low-pass filter with a cutoff frequency of about 30 mHz.…”