Probability Learning11Preparation of this review was supported in part by Contract Nonr 908(16) between the Office of Naval Research and Indiana University. Reproduction in whole or in part is permitted for any purpose of the United States Government.

Estes, W. K.

doi:10.1016/b978-1-4832-3145-7.50010-8

Cited by 54 publications

(20 citation statements)

References 92 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That is, in some trials, the decision appears most similar to past experiences from situations from outside the laboratory, in which responding to trends of change was the best alternative (e.g., social interactions with delayed outcomes). As CAT’s trend of change mode induces the negative recency effect in probability learning tasks, this “generalization” conjecture is consistent with Estes’s (1964) claim that the observed negative recency pattern in these tasks is the result of a habit carried over from real life into the lab.…”

Section: Similarity-based Decision Makingsupporting

confidence: 82%

“…This intuition may emerge from the reasonable convention to start the development of learning models with the analysis of simple static settings, in which sensitivity to patterns leads to poor decisions. The second claim is that indications of reactions to patterns reflect situation specific habits (Estes, 1964). Both claims lead to the implicit assumption that the effort to capture the distinct sensitivities to patterns will result in models too complex to be useful.…”

Section: Contingent Average Rules and Human Behaviormentioning

confidence: 99%

See 1 more Smart Citation

Reliance on small samples, the wavy recency effect, and similarity-based learning.

Plonsky¹,

Teodorescu²,

Erev³

2015

Psychological Review

156

163

View full text Add to dashboard Cite

Many behavioral phenomena, including underweighting of rare events and probability matching, can be the product of a tendency to rely on small samples of experiences. Why would small samples be used, and which experiences are likely to be included in these samples? Previous studies suggest that a cognitively efficient reliance on the most recent experiences can be very effective. We explore a very different and more cognitively demanding process explaining the tendency to rely on small samples: exploitation of environmental regularities. The first part of our study shows that across wide classes of dynamic binary choice environments, focusing only on experiences that followed the same sequence of outcomes preceding the current task is more effective than focusing on the most recent experiences. The second part of our study examines the psychological significance of these sequence-based rules. It shows that these tractable rules reproduce well-known indications of sensitivity to sequences and predict a nontrivial wavy recency effect of rare events. Analysis of published data supports this wavy recency prediction, but suggests an even wavier effect than these sequence-based rules predict. This pattern, and the main behavioral phenomena documented in basic decisions from experience and probability learning tasks, can be captured with a similarity-based model assuming that people follow sequences of outcomes most of the time but sometimes respond to trends. We conclude with theoretical notes on similarity-based learning.

show abstract

Section: Similarity-based Decision Makingsupporting

confidence: 82%

Section: Contingent Average Rules and Human Behaviormentioning

confidence: 99%

Reliance on small samples, the wavy recency effect, and similarity-based learning.

Plonsky¹,

Teodorescu²,

Erev³

2015

Psychological Review

156

163

View full text Add to dashboard Cite

show abstract

“…Systematic positive and negative recency phenomena can not only be observed in Casinos (e.g., Croson & Sundali, 2005) but in stimulus-response learning tasks in various learning domains, often studied under different labels. In probability learning studies (with early roots in classical conditioning) the outlined phenomena are sometimes discussed in terms of Gamblers Fallacy (e.g., when predicting whether the next card from a deck will be red or black; for reviews see Estes, 1964;Wagenaar, 2016), but also under Perruchet effect in stimulus-response conditioning tasks (when predicting whether a puff of nitrogen admitted to they eye will follow a light signal; for a review see Perruchet, 2015), or as mentioned, under labels of positive recency (standard RL like) and negative recency (choose red after observing black three times; CATEGORIES, REWARDS AND SEQUENTIAL PATTERNS 4 see Myers, 2014) also in category learning (e.g., M. Jones, Curran, Mozer, & Wilder, 2013;M. Jones, Love, & Maddox, 2006).…”

Section: Introductionmentioning

confidence: 99%

“…Here, an observed outcome sequence might be RBBBB, and the term wavy recency reflects a tendency of choosing R after observing R then B again after observing B (positive recency), then R after observing B three times (negative recency), and when the B streak continues, choosing B again (delayed positive recency). The most general class of behavior subsuming this pattern, has been called frequency or probability matching (PM), which is more agnostic regarding the exact pattern, but simply summarizes that people tend to enact the observed frequency of events (for reviews, see Estes, 1964;Koehler & James, 2014). If correct predictions are rewarded, as in reward learning, this CATEGORIES, REWARDS AND SEQUENTIAL PATTERNS 5 type of behavior yields a lower payoff than always predicting B if the outcome-generating process was truly random.…”

Section: Introductionmentioning

confidence: 99%

Towards re-unifying the theoretical perspectives on conditioning, category learning, probability learning and economic decisions from experience.

Schlegelmilch,

Wills,

von Helversen

2023

Preprint

View full text Add to dashboard Cite

Stimulus classification is an everyday feat (e.g., in medical diagnoses by differentiating ultrasound images). Category feedback, however, is often non-deterministic (e.g., by 25% chance untrue a.k.a. probabilistic feedback) rendering experiences as somewhat unreliable. In probability learning and economic decisions (when humans try to predict which of two outcomes is more rewarding), it is often observed that humans decide non-rational (probability matching, Gamblers Fallacy). Despite shared origins, the research areas of category learning, probability learning, conditioning and economic decisions (experience based) did not arrive at a consensus of what drives such probability matching strategies. Here, we offer a domain-general integrative model that predicts the mixture of behavioral trends when participants learn probabilistic stimulus-outcome regularities. We use the Category Abstraction Learning framework (CAL), implementing the hypothesis, that humans count the streak of events, and generate simple rules and conditional hypotheses with them. We present simulations of four studies, one from each domain, showing that CAL’s learning mechanisms accurately predict systematic and individual differences in category (correlation) learning, reward learning, risky gambles, and fear conditioning, concerning the phenomena of Gamblers Fallacy, positive and negative recency, or win-stay-lose-shift strategies, in a single model. One central novel CAL hypothesis to link learning phenomena under gains, losses and neutral feedback, is that gains and losses differently affect cognitive control during learning (corrective feedback processing), thereby also providing a novel perspective on risk-preferences in experience-based risky gambles. We discuss CAL’s potential as a domain-general account of human learning in experience-based decisions in light of a broader range of theories from multiple domains.

show abstract

“…Maximization remains the best strategy even when outcome probabilities are not explicit. Many repeated choice paradigms are probability learning tasks with stationary and statistically independent outcome probabilities (see, e.g., Estes, 1964). Despite the apparent simplicity of the problem, people often need hundreds of choice trials and extensive feedback to eventually adopt and maintain a maximizing strategy (e.g., Newell, Koehler, James, Rakow, & van Ravenzwaaij, 2013; Newell & Rakow, 2007; Shanks, Tunney, & McCarthy, 2002).…”

mentioning

confidence: 99%

Hold it! The influence of lingering rewards on choice diversification and persistence.

Schulze¹,

Ravenzwaaij²,

Newell³

2017

Journal of Experimental Psychology: Learning, Memory, and Cogni

View full text Add to dashboard Cite

Learning to choose adaptively when faced with uncertain and variable outcomes is a central challenge for decision makers. This study examines repeated choice in dynamic probability learning tasks in which outcome probabilities changed either as a function of the choices participants made or independently of those choices. This presence/absence of sequential choice-outcome dependencies was implemented by manipulating a single task aspect between conditions: the retention/withdrawal of reward across individual choice trials. The study addresses how people adapt to these learning environments and to what extent they engage in 2 choice strategies often contrasted as paradigmatic examples of striking violation of versus nominal adherence to rational choice: diversification and persistent probability maximizing, respectively. Results show that decisions approached adaptive choice diversification and persistence when sufficient feedback was provided on the dynamic rules of the probabilistic environments. The findings of divergent behavior in the 2 environments indicate that diversified choices represented a response to the reward retention manipulation rather than to the mere variability of outcome probabilities. Choice in both environments was well accounted for by the generalized matching law, and computational modeling-based strategy analyses indicated that adaptive choice arose mainly from reliance on reinforcement learning strategies. (PsycINFO Database Record

show abstract

Probability Learning11Preparation of this review was supported in part by Contract Nonr 908(16) between the Office of Naval Research and Indiana University. Reproduction in whole or in part is permitted for any purpose of the United States Government.

Cited by 54 publications

References 92 publications

Reliance on small samples, the wavy recency effect, and similarity-based learning.

Reliance on small samples, the wavy recency effect, and similarity-based learning.

Towards re-unifying the theoretical perspectives on conditioning, category learning, probability learning and economic decisions from experience.

Hold it! The influence of lingering rewards on choice diversification and persistence.

Contact Info

Product

Resources

About