Order and Chaos in Fixed‐interval Schedules of Reinforcement

Hoyert, Mark Sudlow

doi:10.1901/jeab.1992.57-339

Cited by 31 publications

(17 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…f(t) = k χ t c 3 χ t b 2 [17] where !F(t) was the probability that a reward would be taken after it was available. Here, the t c was again a discrete counter which represented the time after the presentation of a reward.…”

Section: Appendixmentioning

confidence: 99%

Non-Linear Dynamics of Operant Behavior: A New Approach via the Extended Return Map

Huston²

2002

Reviews in the Neurosciences

View full text Add to dashboard Cite

Previous efforts to apply non-linear dynamic tools to the analysis of operant behavior revealed some promise for this kind of approach, but also some doubts, since the complexity of animal behavior seemed to be beyond the analyzing ability of the available tools. We here outline a series of studies based on a novel approach. We modified the so-called 'return map' and developed a new method, the 'extended return map' (ERM) to extract information from the highly irregular time series data, the inter-response time (IRT) generated by Skinner-box experiments. We applied the ERM to operant lever pressing data from rats using the four fundamental reinforcement schedules: fixed interval (FI), fixed ratio (FR), variable interval (VI) and variable ratio (VR). Our results revealed interesting patterns in all experiment groups. In particular, the FI and VI groups exhibited well-organized clusters of data points. We calculated the fractal dimension out of these patterns and compared experimental data with surrogate data sets, that were generated by randomly shuffling the sequential order of original IRTs. This comparison supported the finding that patterns in ERM reflect the dynamics of the operant behaviors under study. We then built two models to simulate the functional mechanisms of the FI schedule. Both models can produce similar distributions of IRTs and the stereotypical 'scalloped' curve characteristic of FI responding. However, they differ in one important feature in their formulation: while one model uses a continuous function to describe the probability of occurrence of an operant behavior, the other one employs an abrupt switch of behavioral state. Comparison of ERMs showed that only the latter was able to produce patterns similar to the experimental results, indicative of the operation of an abrupt switch from one behavioral state to another over the course of the inter-reinforcement period. This example demonstrated the ERM to be a useful tool for the analysis of IRT accompanying intermittent reinforcement schedules and for the study of the non-linear dynamics of operant behavior.

show abstract

Section: Appendixmentioning

confidence: 99%

Non-Linear Dynamics of Operant Behavior: A New Approach via the Extended Return Map

Huston²

2002

Reviews in the Neurosciences

View full text Add to dashboard Cite

show abstract

“…Recent forays into behavioral dynamics, including models based on the sequential structure of responding (e.g., Hoyert, 1992;Palya, 1992) or on linear-systems analysis (e.g., McDowell, Bass, & Kessel, 1992) suggest potential starts. That subjects are capable of discriminating sequential structure in environmental events as well as in behavior should come as no surprise-the areas of psychophysics dealing with topics such as timing (e.g., Gibbon & Allan, 1984), numerosity (e.g., Gallistel, 1989), and so forth are replete with such demonstrations.…”

Section: Discussionmentioning

confidence: 99%

Response Acquisition Under Targeted Percentile Schedules: A Continuing Quandary for Molar Models of Operant Behavior

Galbicka

Kautz

Jagers

1993

J Exper Analysis Behavior

View full text Add to dashboard Cite

The number of responses rats made in a "run" of consecutive left-lever presses, prior to a trial-ending right-lever press, was differentiated using a targeted percentile procedure. Under the nondifferential baseline, reinforcement was provided with a probability of .33 at the end of a trial, irrespective of the run on that trial. Most of the 30 subjects made short runs under these conditions, with the mean for the group around three. A targeted percentile schedule was next used to differentiate run length around the target value of 12. The current run was reinforced if it was nearer the target than 67% of those runs in the last 24 trials that were on the same side of the target as the current run. Programming reinforcement in this way held overall reinforcement probability per trial constant at .33 while providing reinforcement differentially with respect to runs more closely approximating the target of 12. The mean run for the group under this procedure increased to approximately 10. Runs approaching the target length were acquired even though differentiated responding produced the same probability of reinforcement per trial, decreased the probability of reinforcement per response, did not increase overall reinforcement rate, and generally substantially reduced it (i.e., in only a few instances did response rate increase sufficiently to compensate for the increase in the number of responses per trial). Models of behavior predicated solely on molar reinforcement contingencies all predict that runs should remain short throughout this experiment, because such runs promote both the most frequent reinforcement and the greatest reinforcement per press. To the contrary, 29 of 30 subjects emitted runs in the vicinity of the target, driving down reinforcement rate while greatly increasing the number of presses per pellet. These results illustrate the powerful effects of local reinforcement contingencies in changing behavior, and in doing so underscore a need for more dynamic quantitative formulations of operant behavior to supplement or supplant the currently prevalent static ones.

show abstract

“…The learning curve is typically drawn as a smooth function. There is actually a lot of irregularity in the portion of the curve prior to the asymptote (Hoyert, 1992). The neurological explanation is that neural firing patterns are themselves chaotic in the early phases of learning while the brain is testing out possible synaptic pathways.…”

Section: Learningmentioning

confidence: 99%

Nonlinear Dynamical Systems Applications to Psychology and Management

Guastello

2011

The Sage Handbook of Complexity and Management

View full text Add to dashboard Cite

This chapter surveys the recent developments in the application of nonlinear dynamical systems (NDS) theory to theoretical and practical problems encountered in psychology that are also relevant to management. For the benefit of non-psychologists, it is important to note that the scope of psychology is expansive. Introductory textbooks are typically organized around the following themes: brain physiology and behavior, psychophysics, sensation, perception, learning, memory, cognition, intelligence and mental measurement, development, social psychology, motivation and emotion, personality of normal range people, abnormal psychology, psychotherapy and counseling, and industrial-organizational psychology. At the other end of the professional spectrum, the largest professional organization for psychologists, the American Psychological Association, contains more than 50 topical interest groups in addition to its general membership core. The literature on NDS psychology reaches all the major areas of psychology and is

show abstract

Order and Chaos in Fixed‐interval Schedules of Reinforcement

Cited by 31 publications

References 28 publications

Non-Linear Dynamics of Operant Behavior: A New Approach via the Extended Return Map

Non-Linear Dynamics of Operant Behavior: A New Approach via the Extended Return Map

Response Acquisition Under Targeted Percentile Schedules: A Continuing Quandary for Molar Models of Operant Behavior

Nonlinear Dynamical Systems Applications to Psychology and Management

Contact Info

Product

Resources

About