Further Perceptions of Probability: In Defence of Associative Models – A Commentary on Gallistel et al. 2014

Abstract

25Extensive research in the behavioural sciences has addressed people's ability to learn 26 probabilities of stochastic events, typically assuming them to be stationary (i.e., constant over 27 time). Only recently have there been attempts to model the cognitive processes whereby people 28 learnand tracknon-stationary probabilities, reviving the old debate on whether learning 29 occurs trial-by-trial or by occasional shifts between discrete hypotheses. Trial-by-trial updating 30 modelssuch as the delta-rule modelhave been popular in describing human learning in 31 various contexts, but it has been argued that they are inadequate for explaining how humans 32 update beliefs about non-stationary probabilities. Specifically, it has been claimed that these 33 models cannot account for the discrete, stepwise updating that characterises response patterns 34 in experiments where participants tracked a non-stationary probability based on observed 35 outcomes. Here, we demonstrate that the rejection of trial-by-trial models was premature for 36 two reasons. First, our experimental data suggest that the stepwise behaviour depends on details 37 of the experimental paradigm. Hence, discreteness in response data does not necessarily imply 38 discreteness in internal belief updating. Second, previous studies have dismissed trial-by-trial 39 models mainly based on qualitative arguments rather than quantitative model comparison. To 40 evaluate the models more rigorously, we performed a likelihood-based model comparison 41 between stepwise and trial-by-trial updating models. Across eight datasets collected in three 42 different labs, human behaviour is consistently best described by trial-by-trial updating models. 43Our results suggest that trial-by-trial updating plays a prominent role in the cognitive processes 44 underlying learning of non-stationary probabilities. 45 46

Individual Differences in the Perception of Probability

“…Rather than adjusting the slider position modestly with each new piece of evidence, subjects report a new estimate only periodically. Infrequent adjustments in this paradigm might then be interpreted as reflecting implicit integration of new observations, updating internal estimates without overt adjustment [26]. In this view, subjects who produce repulsive estimates appear to have lower thresholds for updating their declared estimates of the hidden state (analogous to the definition of overconfidence by Moore and Healy [31]).…”

“…Our results regarding the coexistence of a range of biases across subjects completing the same task complement those of Zhang, Ren, and Maloney [25], who find that the type of bias may change across tasks for the same participant. Varying levels of individual accuracy have been reported in the same kind of task by Forsgren, Juslin, and Van Den Berg [26] but not analyzed in detail in this respect.…”

“…The unconditional frequency of adjustment is 8.8 percent, despite the steady stream of new ring realizations and the fact that the Bayesian forecasts adjust, however modestly, in response to each new ring draw. Infrequent adjustment often arises in experiments that feature long series of repeated trials [16,26]. While not the focus of our study, it needs to be taken into account, to avoid mischaracterizing subjective biases.…”

“…We first consider the basic formulation of a delta rule model, a model that is widely used in the literature, including specifically in the literature on change-point estimation [26,35]. Error-based updating also represents a computation characteristic of midbrain dopaminergic circuits [36,37]; incremental updating based on unexpected error signals are ubiquitous in descriptions of action values learned through reinforcement [38].…”

How do people predict a random walk? Lessons for models of human cognition

“…One of the most notable approaches to iterative predictions is provided by associative learning methods, where expectations are captured psychologically via mental associations between relevant stimuli: these connections are updated with experience according to errors in anticipated events, strengthening where unexpected outcomes occur and weakening when expected outcomes fail to appear (Bush & Mosteller, 1951;Rescorla & Wagner, 1972;Pearce & Hall, 1980;Pearce & Bouton, 2001). In the case of basic numerical prediction tasks such as that considered here, such techniques can be directly applied to continuous estimates, using anticipation errors to update predicted values themselves, for example in estimates of probabilities (Behrens, Woolrich, Walton, & Rushworth, 2007;Forsgren, Juslin, & Van Den Berg, 2020). Similar error-based learning techniques also appear in depictions of market predictions in economics, in this case being described as adaptive expectations models (Hey, 1994;Hommes, 2011;Afrouzi, Kwon, Landier, Ma, & Thesmar, 2020).…”