A unified explanation of variability and bias in human probability judgments: How computational noise explains the mean-variance signature

Abstract

Human probability judgments are variable and subject to systematic biases. Sampling-based accounts of probability judgment have successfully explained such idiosyncrasies by assuming that people remember or simulate instances of events and base their judgments on sampled frequencies. Biases have been explained either by noise corrupting sample accumulation (the Probability Theory + Noise account), or as a Bayesian adjustment to the uncertainty implicit in small samples (the Bayesian sampler). While these two accounts closely mimic one another, here we show that they can be distinguished by a novel linear regression method that relates the variance of repeated judgments to their means. First, the efficacy of the method is confirmed by model recovery, and it more accurately recovers parameters than computationally complex methods. Second, the method is applied to both existing and new probability judgment data, which confirm that judgments are based on a small number of samples that are adjusted by a prior, as predicted by the Bayesian sampler.

Noise in Cognition: Bug or Feature?

“…However, in a task in which participants were repeatedly asked to estimate the relative frequency of geometric objects on a screen, the variability of probability judgments peaked for probability estimates in the center of the range (i.e., close to 0.5), and was well described by a binomial distribution instead of a Gaussian distribution (Howe & Costello, 2020). While these results though were based on perceptual stimuli for which sensory noise could play a role, a later study found the same results when participants were estimating probabilities from memory, e.g., estimating the probability of rain on a random day in England, and also showed that additive Gaussian noise could not explain the inverted-u shape, even if responses were censored at the edges of the response range (see Figure 3A 1 and Figure 3A 2 ; Sundh et al, 2021).…”

“…But substantial variability also occurs in tasks in which perceptual acuity is no barrier. For example, people's judgments of the probability of the same event vary from one occasion to another, even when no new information has been observed between judgments (Sundh et al, 2021). Even people's preferences between monetary gambles will show considerable variability over short periods of time (Loomes & Sugden, 1998;Mosteller & Nogee, 1951;.…”

“…Individual behavior is variable: participants change their responses even when facing the same choice or judgment without new information (e.g., Mosteller & Nogee, 1951;Sundh et al, 2021). While this has often been attributed to sensory or motor noise that is presumed to sit outside the deterministic core of cognition, here we look at studies across two domains of investigation that have come to a different conclusion.…”

“…If the counts are adjusted for the autocorrelation in the samples, then the counts will follow a binomial distribution that produces an inverse-u shaped relationship between mean probability judgments of an event and the variance of the judgments of that event, as shown in the human data (see Figure 3A 3 ). Furthermore, the fact that the mean-variance curve is falls to zero before reaching the edge of the range can be explained by a prior over responses: knowledge that probabilities tend not to always be zero or one (Sundh et al, 2021).…”

Comparing Probabilistic Accounts of Probability Judgments

“…For small N, both models predict only a limited responses across trials on their mean response for each query type. Their findings suggest that earlier fits with Wasserstein distance may have produced biased results (Sundh et al, 2021). They reported evidence for the truncation of responses and a correlation between variance and shrinkage parameter estimates across participants.…”