Visualizing Uncertainty for Non-Expert End Users: The Challenge of the Deterministic Construal Error

Abstract

There is a growing body of evidence that numerical uncertainty expressions can be used by non-experts to improve decision quality. Moreover, there is some evidence that similar advantages extend to graphic expressions of uncertainty. However, visualizing uncertainty introduces challenges as well. Here, we discuss key misunderstandings that may arise from uncertainty visualizations, in particular the evidence that users sometimes fail to realize that the graphic depicts uncertainty. Instead they have a tendency to interpret the image as representing some deterministic quantity. We refer to this as the deterministic construal error. Although there is now growing evidence for the deterministic construal error, few studies are designed to detect it directly because they inform participants upfront that the visualization expresses uncertainty. In a natural setting such cues would be absent, perhaps making the deterministic assumption more likely. Here we discuss the psychological roots of this key but underappreciated misunderstanding as well as possible solutions. This is a critical question because it is now clear that members of the public understand that predictions involve uncertainty and have greater trust when uncertainty is included. Moreover, they can understand and use uncertainty predictions to tailor decisions to their own risk tolerance, as long as they are carefully expressed, taking into account the cognitive processes involved.

Examining Effort in 1D Uncertainty Communication Using Individual Differences in Working Memory and NASA-TLX

“…With the rising interest in uncertainty communication, researchers have conducted a growing number of empirical studies to identify which uncertainty visualization techniques best support decisions with uncertain data (for reviews, see [40,70]). This research provides evidence both that uncertainty visualizations support numerous types of judgments (e.g., [16,17,39,50,54,83]) and that uncertainty visualizations can produce systematic biases (e.g., [5,43,72,74]). Recently, however, prominent scholars in the field have questioned the utility of uncertainty visualizations compared to textual expressions of uncertainty [43].…”

“…This research provides evidence both that uncertainty visualizations support numerous types of judgments (e.g., [16,17,39,50,54,83]) and that uncertainty visualizations can produce systematic biases (e.g., [5,43,72,74]). Recently, however, prominent scholars in the field have questioned the utility of uncertainty visualizations compared to textual expressions of uncertainty [43]. As criticisms, they cite studies that find no difference between textual and visualized expressions of uncertainty (e.g., [42,59,62]), differences that are ameliorated with a longer time to complete the task [14], and biases that uncertainty visualizations produce (e.g., [5,43,62,69,72,85]).…”

“…Recently, however, prominent scholars in the field have questioned the utility of uncertainty visualizations compared to textual expressions of uncertainty [43]. As criticisms, they cite studies that find no difference between textual and visualized expressions of uncertainty (e.g., [42,59,62]), differences that are ameliorated with a longer time to complete the task [14], and biases that uncertainty visualizations produce (e.g., [5,43,62,69,72,85]). Given that situations exist where the advantages of uncertainty visualizations become less clear and that some uncertainty visualizations produce biases, these researchers postulate that textual expressions of uncertainty may be preferable in certain cases [43].…”

Impact of COVID-19 Forecast Visualizations on Pandemic Risk Perceptions

“…The stimuli were generated using COVID-19 Fore-castHub created by the Reich Lab from the University of Massachusetts Amherst, available at viz.covid19forecasthub.org. conventional uncertainty visualizations, such as intervals, can be confusing or lead to misinterpretations of the data (11)(12)(13)(14)(15)(16)(17)(18)(19), even for experts (20), and controlling for education (21). For example, the Cone of Uncertainty, produced by the National Hurricane Center, shows a 66% confidence interval around the storm's predicted path.…”

“…This misinterpretation persists even after training on interpreting the forecast correctly (22) (see also, 11). Indeed, both novices (12)(13)(14)(15)(16)(17)(18)(19)21) and published researchers in psychology, neuroscience, and medicine misinterpret confidence intervals (20) (for reviews of errors in uncertainty visualization, see 13,23).…”