The Bayesian sampler: Generic Bayesian inference causes incoherence in human probability judgments.

Abstract

Human probability judgments are systematically biased, in apparent tension with Bayesian models of cognition. But perhaps the brain does not represent probabilities explicitly, but approximates probabilistic calculations through a process of sampling, as used in computational probabilistic models in statistics. Naïve probability estimates can be obtained by calculating the relative frequency of an event within a sample, but these estimates tend to be extreme when the sample size is small. We propose instead that people use a generic prior to improve the accuracy of their probability estimates based on samples, and we call this model the Bayesian sampler. The Bayesian sampler trades off the coherence of probabilistic judgments for improved accuracy, and provides a single framework for explaining phenomena associated with diverse biases and heuristics such as conservatism and the conjunction fallacy. The approach turns out to provide a rational reinterpretation of "noise" in an important recent model of probability judgment, the probability theory plus noise model (Costello & Watts, 2014, 2016a, 2019Costello, Watts, & Fisher, 2018), making equivalent average predictions for simple events, conjunctions, and disjunctions. The Bayesian sampler does, however, make distinct predictions for conditional probabilities, and we show in a new experiment that this model better captures these judgments both qualitatively and quantitatively.

Broken Physics: A Conjunction-Fallacy Effect in Intuitive Physical Reasoning

“…However, such theories might be modified in light of these results. For example, rather than aggregating across multiple simulation runs to make a probabilistic inference, people might use some type of biased aggregation scheme that results in judgment errors (Zhu, Sanborn, & Chater, 2020). It remains to be seen whether this biased aggregation approach can provide a simultaneous account of all the other documented phenomena in intuitive physical reasoning.…”

Probabilistic Biases Meet the Bayesian Brain

“…Zhu et al (2020) showed that a number of identical mean predictions can be derived from an even more “rational” model, the Bayesian sampler . Zhu et al’s starting point is that with small samples, reading off relative frequencies cannot be quite right.…”

“…These reliable matches and mismatches between human judgment and probability theory form a challenge to nonsampling models of probability distortion; Costello and Watts (2014 have shown how a sampling model captures both the distortions and the patterns in human probability judgments-demonstrating that these judgments are, as they put it, "surprisingly rational" after all and that irrational judgments are the result of noise. Zhu et al (2020) showed that a number of identical mean predictions can be derived from an even more "rational" model, the Bayesian sampler. Zhu et al's starting point is that with small samples, reading off relative frequencies cannot be quite right.…”

“…The overlaid dots are best-fitting-model predictions for three models (green squares: a baseline relative-frequency model, i.e., simply the proportion of samples with a particular property; blue triangles: the probability-theory-plus-noise model, developed by Watts, 2014, 2016; red dots: the Bayesian sampler). This figure is adapted from Zhu, Sanborn, and Chater (2020). such unification might be.…”

“…There is, however, an important distinction between cognitive models that apply "raw" relative frequencies, based on a mental sample (e.g., Costello & Watts, 2014), and those that take a Bayesian approach a step further (e.g., Zhu, Sanborn, & Chater, 2020) by integrating samples with background knowledge (in the spirit of Bayesian Monte Carlo in machine learning; Rasmussen & Ghahramani, 2003). This difference is especially important when sample sizes are small.…”