Suboptimal human inference inverts the bias-variance trade-off for decisions with asymmetric evidence

Abstract

Decisions based on rare events are challenging because rare events alone can be both informative and unreliable as evidence. How humans should and do overcome this challenge is not well understood. Here we present results from a preregistered study of 200 on-line participants performing a simple inference task in which the evidence was rare and asymmetric but the priors were symmetric. Consistent with a Bayesian ideal observer, most participants exhibited choice asymmetries that reflected a tendency to rationally interpret a rare event as evidence for the alternative likely to produce slightly more events, even when the two alternatives were equally likely a priori. A subset of participants exhibited additional biases based on an under-weighing of rare events. The results provide new quantitative and theoretically grounded insights into rare-event inference, which is relevant to both real-world problems like predicting stock-market crashes and common laboratory tasks like predicting changes in reward contingencies.

Normative decision rules in changing environments

“…For example, if a normative analysis of performance on a dynamic reward task produces threshold dynamics similar to those in Figure 2B , then the fitting library should include a piecewise-constant threshold (or urgency signal) model. Combining these model-based investigations with model-free approaches, such as rate-distortion theory ( Berger, 2003 ; Eissa et al, 2021 ), can also aid in identifying commonalities in performance and resource usage within and across model classes without the need for pilot experiments.…”

Normative Decision Rules in Changing Environments

“…For example, if a normative analysis of performance on a dynamic reward task produces threshold dynamics similar to those in Figure 2 B , then the fitting library should include a piecewise-constant threshold (or urgency signal) model. Combining these model-based investigations with model-free approaches, such as rate-distortion theory ( Berger, 2003; Eissa et al, 2021 ), can also aid in identifying commonalities in performance and resource usage within and across model classes without the need for pilot experiments.…”

“…For example, if a normative analysis of performance on a dynamic reward task produces threshold dynamics similar to those in Figure2B, then the fitting library should include a piecewise-constant threshold (or urgency signal) model. Combining these model-based investigations with model-free approaches, such as rate-distortion theory(Berger, 2003;Eissa et al, 2021), can also aid in identifying commonalities in performance and resource usage within and across model classes without the need for pilot experiments.Our work complements the existing literature on optimal decision thresholds by demonstrating the prevalence of behaviors reflective of non-monotonic decision thresholds. Most studies describing decision strategies with time-varying decision thresholds focus on environments with fixed structure, in which dynamic decision thresholds are adapted as the observer acquires knowledge of the environment.…”