Decision Making for Risk Management: A Comparison of Graphical Methods for Presenting Quantitative Uncertainty

Abstract

Previous research has shown that people err when making decisions aided by probability information. Surprisingly, there has been little exploration into the accuracy of decisions made based on many commonly used probabilistic display methods. Two experiments examined the ability of a comprehensive set of such methods to effectively communicate critical information to a decision maker and influence confidence in decision making. The second experiment investigated the performance of these methods under time pressure, a situational factor known to exacerbate judgmental errors. Ten commonly used graphical display methods were randomly assigned to participants. Across eight scenarios in which a probabilistic outcome was described, participants were asked questions regarding graph interpretation (e.g., mean) and made behavioral choices (i.e., act; do not act) based on the provided information indicated that decision-maker accuracy differed by graphical method; error bars and boxplots led to greatest mean estimation and behavioral choice accuracy whereas complementary cumulative probability distribution functions were associated with the highest probability estimation accuracy. Under time pressure, participant performance decreased when making behavioral choices.

Guidance on Communication of Uncertainty in Scientific Assessments

“…However, similar to Ibrekk and Morgan (), Edwards et al. () found that background knowledge in statistics or familiarity with a display method did not influence accuracy.…”

“…The results of Edwards et al. () indicated that error bars and box plots were the most accurate for estimating the mean. Participants rendered more accurate means when using a graphical display method that explicitly provided the necessary information.…”

“…You may consider simultaneously using numbers, verbal descriptions and graphs, more or less detailed information in the same document or in different (e.g. background) documents or communication platforms (based on Spiegelhalter et al., ; Edwards et al., ; Kuhn, ; Dieckmann et al., ). Provide information on uncertainty in full as soon as it is identified. Information about what is being carried out to reduce uncertainty should be included (Frewer et al., ).…”

“… For highlighting the mean, using a CDF alone (without indicating the mean and without explanation) is not a good way to communicate (Ibrekk and Morgan, ). CDFs and multivariate density function graphs should be used with caution (Edwards et al., ). A CDF plotted directly above a PDF with the same horizontal scale, and with the location of the mean clearly marked on both curves, should be used (Ibrekk and Morgan, ). CCDFs should be preferred for communicating precise probabilities (Edwards et al., ). Pie charts to represent uncertainty (discretised PDFs) show potential for confusion and should be avoided (even if many people declared that they prefer this representation) (Ibrekk and Morgan, ). …”

Simulation-based decision support for evaluating operational plans

“…Dispersion of data can be due to measurement errors or underlying stochastic behavior, such as the Monte Carlo process in our simulation. Edwards et al [54] discuss different ways to visualize dispersions of measurement data; the uncertainty dispersion implies how people tend to interpret the uncertainties depending on how they are visually represented, e.g., as box plots, error bars, scatterplots or various probability density distributions. In general, box plots are easy to understand and seem to give a better match between real dispersion and the interpretation the viewer gets.…”