Error Bars Considered Harmful: Exploring Alternate Encodings for Mean and Error

Abstract

When making an inference or comparison with uncertain, noisy, or incomplete data, measurement error and confidence intervals can be as important for judgment as the actual mean values of different groups. These often misunderstood statistical quantities are frequently represented by bar charts with error bars. This paper investigates drawbacks with this standard encoding, and considers a set of alternatives designed to more effectively communicate the implications of mean and error data to a general audience, drawing from lessons learned from the use of visual statistics in the information visualization community. We present a series of crowd-sourced experiments that confirm that the encoding of mean and error significantly changes how viewers make decisions about uncertain data. Careful consideration of design tradeoffs in the visual presentation of data results in human reasoning that is more consistently aligned with statistical inferences. We suggest the use of gradient plots (which use transparency to encode uncertainty) and violin plots (which use width) as better alternatives for inferential tasks than bar charts with error bars.

Fair Statistical Communication in HCI

“…HCI and infovis researchers also have the exciting opportunity to contribute new research, for example by studying new visual representations for communicating study results (Correll and Gleicher, 2014;Perin et al, 2014). Such representations do not have to be static, and there are many ways computers, animation, and interactivity can be used to teach statistics and convey scientific evidence to a wide audience (Victor, 2011).…”

“…The scarcity of graphical standards should be taken as an opportunity to explore custom visual designs, within the limits suggested by Tip 18. For example, there are many options for displaying error bars (Figure 9): while the design (1) is widely used, (2) is a common alternative that has the advantage of de-emphasizing confidence limits; The variant (3) improves the legibility of point estimates; Error bars can also be combined with bar charts (4); Bars help compare magnitudes on a ratio scale (Zacks and Tversky, 1999), but they introduce visual asymmetry (Newman and Scholl, 2012;Correll and Gleicher, 2014) and tend to de-emphasize error bars; This is evident in the so-called "dynamite plots" (5); The design (6) supports ratio comparison while maintaining emphasis on error bars; Finally, error bars can be combined with line charts (7) to convey temporal ordering (Zacks and Tversky, 1999) or within-subject factors (Cumming, 2012, p.172). Fig.…”

“…A confidence interval can be thought of as abstracting a continuous "plausibility" function (Cumming, 2012, pp.98-102;Cumming, 2013, p.17). While a recent study has explored alternatives to error bars that visually convey this continuity (Correll and Gleicher, 2014), the classical error bar has the benefit of being visually cleaner and more economical in terms of space and data-ink ratio (Wainer, 1984, p.139). The edges of error bars offer visual reference points, whereas other representations such as color gradients may not.…”

Investigating Representation Alternatives for Communicating Uncertainty to Non-experts

“…Olston et al [8] examined visualizations for presenting bounded uncertainty by adjusting the visual elements and including transparency in the visualizations and Tak et al [13] used seven variations of a line graph to investigate the perceived certainty. Correll and Gleicher [1] proposed a redesign of bar charts and found that less well-known visualizations improve performance for interferential tasks.…”

A crowdsourcing system for integrated and reproducible evaluation in scientific visualization

“…An additional benefit is that participants of crowdsourced user studies can participate at any given time, and is not limited to the opening hours of the lab. While there has not yet been any work published where crowdsourcing has been utilized for evaluating scientific visualization research, crowdsourcing has been successfully utilized for user studies in information visualization [9,10,16,17,19,24,38,47].…”