A Computational Model of Selection by Consequences

McDowell, J. J.

doi:10.1901/jeab.2004.81-297

Cited by 79 publications

(147 citation statements)

References 21 publications

Supporting

Mentioning

142

Contrasting

Unclassified

Order By: Relevance

“…Variability describes the class of potential behaviors, selection describes the potentiating effects of reinforcement on behavior, and retention describes the physiological changes that permit maintenance of adaptive responses. McDowell (2004) developed a model of instrumental conditioning based on such evolutionary principles. He represented actions as ‘populations’ of behavior.…”

Section: Discussionmentioning

confidence: 99%

Navigating complex decision spaces: Problems and paradigms in sequential choice.

Walsh¹,

Anderson²

2014

Psychological Bulletin

View full text Add to dashboard Cite

To behave adaptively, we must learn from the consequences of our actions. Doing so is difficult when the consequences of an action follow a delay. This introduces the problem of temporal credit assignment. When feedback follows a sequence of decisions, how should the individual assign credit to the intermediate actions that comprise the sequence? Research in reinforcement learning provides two general solutions to this problem: model-free reinforcement learning and model-based reinforcement learning. In this review, we examine connections between stimulus-response and cognitive learning theories, habitual and goal-directed control, and model-free and model-based reinforcement learning. We then consider a range of problems related to temporal credit assignment. These include second-order conditioning and secondary reinforcers, latent learning and detour behavior, partially observable Markov decision processes, actions with distributed outcomes, and hierarchical learning. We ask whether humans and animals, when faced with these problems, behave in a manner consistent with reinforcement learning techniques. Throughout, we seek to identify neural substrates of model-free and model-based reinforcement learning. The former class of techniques is understood in terms of the neurotransmitter dopamine and its effects in the basal ganglia. The latter is understood in terms of a distributed network of regions including the prefrontal cortex, medial temporal lobes cerebellum, and basal ganglia. Not only do reinforcement learning techniques have a natural interpretation in terms of human and animal behavior, but they also provide a useful framework for understanding neural reward valuation and action selection.

show abstract

Section: Discussionmentioning

confidence: 99%

Navigating complex decision spaces: Problems and paradigms in sequential choice.

Walsh¹,

Anderson²

2014

Psychological Bulletin

View full text Add to dashboard Cite

show abstract

“…It is also important to note that all three Darwinian rules ( Figure I, Section III) must operate in order for the theory to function properly. This is obvious for selection and reproduction, but McDowell (2004) found that mutation was also necessary. He reported that without mutation, behavior becomes maladaptively perseverative, that is, it gets stuck in an arbitrary class and becomes unresponsive to environmental resources and to changes in their availability.…”

Section: Discussionmentioning

confidence: 99%

“…A Material Mechanism for the Theory McDowell (2004) suggested that the success of the evolutionary theory "means that the material operation of a biological organism . .…”

Section: Discussionmentioning

confidence: 99%

“…The evolutionary theory of adaptive behavior dynamics consists of a set of rules that operates on a population of potential behaviors (McDowell, 2004). The behaviors are potential in the sense that they may or may not be emitted by a virtual organism that is atiimated by the theory.…”

Section: Rules Of the Evolutionary Theorymentioning

confidence: 99%

“…In addition, at moderate mutation rates (3% to 20%), exponents estimated from fits of Equation 6 varied around 0.8, as has been found in experiments with live organisms. Residuals from the fits of Equations 2 and 6 were tested for linear, quadratic, and cubic polynomial trends, which is a more powerful method of trend testing than the Reich (1992) method used by McDowell (2004). The majority of the fits of Equation 2 (24 of 30), but only two of the fits of Equation 6, yielded nonrandom residuals.…”

Section: Tests Against the Quantitative Criterion Datamentioning

confidence: 99%

See 2 more Smart Citations

A quantitative evolutionary theory of adaptive behavior dynamics.

McDowell¹

2013

Psychological Review

Self Cite

View full text Add to dashboard Cite

The idea that behavior is selected by its consequences in a process analogous to organic evolution has been discussed for over 100 years. A recently proposed theory instantiates this idea by means of a genetic algorithm that operates on a population of potential behaviors. Behaviors in the population are represented by numbers in decimal integer (phenotypic) and binary bit string (genotypic) forms. One behavior from the population is emitted at random each time tick, after which a new population of potential behaviors is constructed by recombining parent behavior bit strings. If the emitted behavior produced a benefit to the organism, then parents are chosen on the basis of their phenotypic similarity to the emitted behavior; otherwise, they are chosen at random. After parent behavior recombination, the population is subjected to a small amount of mutation by flipping random bits in the population's bit strings. The behavior generated by this process of selection, reproduction, and mutation reaches equilibrium states that conform to every empirically valid equation of matching theory, exactly and without systematic error. These equations are known to describe the behavior of many vertebrate species, including humans, in a variety of experimental, naturalistic, natural, and social environments. The evolutionary theory also generates instantaneous dynamics and patterns of preference change in constantly changing environments that are consistent with the dynamics of live-organism behavior. These findings support the assertion that the world of behavior we observe and measure is generated by evolutionary dynamics.

show abstract

Application of the evolutionary theory of behavior dynamics to severe challenging behavior

Hagopian,

Falligant

2023

J of App Behav Analysis

View full text Add to dashboard Cite

The evolutionary theory of behavior dynamics (ETBD) is a genetic algorithm that applies the Darwinian principles of evolutionary biology to model how behavior changes dynamically via selection by contingencies of reinforcement. The ETBD is a complexity theory where low‐level rules of selection, reproduction, and mutation operate iteratively to animate “artificial organisms” that generate emergent outcomes. Numerous studies have demonstrated the ETBD can accurately model behavior of live animals in the laboratory, and it has been applied recently to model automatically maintained self‐injury. The purpose of the current series of studies was to further extend the application of the ETBD to model additional functional classes of challenging behavior and clinical procedures. Outcomes obtained with artificial organisms generally corresponded well with outcomes observed with clinical cases sourced from consecutive controlled case series studies. Conceptual and methodological considerations on the application of the ETBD to model challenging behavior are discussed.

show abstract

A Computational Model of Selection by Consequences

Cited by 79 publications

References 21 publications

Navigating complex decision spaces: Problems and paradigms in sequential choice.

Navigating complex decision spaces: Problems and paradigms in sequential choice.

A quantitative evolutionary theory of adaptive behavior dynamics.

Application of the evolutionary theory of behavior dynamics to severe challenging behavior

Contact Info

Product

Resources

About