Hypothetical Outcome Plots Outperform Error Bars and Violin Plots for Inferences about Reliability of Variable Ordering

Abstract

Many visual depictions of probability distributions, such as error bars, are difficult for users to accurately interpret. We present and study an alternative representation, Hypothetical Outcome Plots (HOPs), that animates a finite set of individual draws. In contrast to the statistical background required to interpret many static representations of distributions, HOPs require relatively little background knowledge to interpret. Instead, HOPs enables viewers to infer properties of the distribution using mental processes like counting and integration. We conducted an experiment comparing HOPs to error bars and violin plots. With HOPs, people made much more accurate judgments about plots of two and three quantities. Accuracy was similar with all three representations for most questions about distributions of a single quantity.

How Data Workers Cope with Uncertainty

“…Ignoring Uncertainty: Although seven participants declared having ignored a known uncertainty in their data at some point during the analysis, this is not a common strategy (P2, 4,5,6,7,11,12). Our participants ignored known uncertainties when the category of uncertainty itself was not relevant to the analysis or when dealing with it was outside the scope of their role or expertise.…”

“…Visualization tools were used by nine participants to support different tasks, e.g. P2 used Jigsaw [24] do nothing (P7) C: annotate to prevent data usage of erroneous records (P7) M: filter uncertain data; M: delete model that generates errors; A: improve acquisition source and tools; A: acquire data from multiple sources; R: group discussions^M: manual correction; C: set task constraints (P1, 3,5,6,7,8,11,12) Imprecision (8 participants) do nothing (P5) C: model uncertainty ; C: annotate data quality, confidence or possible range; R: compare to literature^C: set quality threshold (P1,2,9,12)…”

“…M: filter imprecise data; M: enrich with better quality data^A: improve acquisition^R: human expertise; M: aggregate data; R: prevent error propagation (P1, 3,7,9,11,12) Noise (4 participants) f P: plot data with uncertainty (P9) R: discuss with experts to identify noise^M: remove noise; C: compute threshold level^M: remove noise (P4,6,9,11) Table 3. Data uncertainty coping strategies by high level goals: Ignore, Understand and Minimise (Exploit is not shown as it only applies to missing data & imprecision).…”

“…When high level tasks are used, few authors justify their choice [19]. Recent reviews [19,18,3] on the exploration of uncertain data show an existing, large body of work that focuses on communication [17,12], collaboration [35,8] and user evaluations [4,34]. These reviews also highlight the need to focus on enabling reasoning and sense-making under uncertainty [25].…”

Capture & Analysis of Active Reading Behaviors for Interactive Articles on the Web

“…HopScroll plots user scroll positions over time to reveal a diversity of reading behaviors. In addition to an aggregate overview, HopScroll shows individual user sessions in an animated fashion akin to Hypothetical Outcome Plots [HRA15]. Readuction uses dimensionality reduction to visually cluster related feature vectors automatically derived from event logs.…”