The Effect of Uncertainty Visualizations on Decision Making in Weather Forecasting

Abstract

People's reasoning with uncertainty information is often flawed. Visual representations can help, but little is known about what is the best way to present such information. Two studies investigated the effect of visualizations on the understanding and use of wind speed forecast uncertainty. Participants varied in expertise from novices in weather forecasting (Experiment 1) to professional forecasters (Experiment 2). The authors investigated three visualizations: (a) a chart showing the amount of uncertainty, (b) a chart showing the worst-case scenario, and (c) a box plot of likely wind speeds. Participants were asked to determine the relative uncertainty in the forecast, predict wind speed, and decide whether to post a high-wind warning advisory. The results for novices and professional forecasters were similar. The uncertainty chart enhanced awareness of the degree of uncertainty associated with the forecast, box plots improved reading accuracy, and presenting a visualization of the worst-case scenario introduced bias in the deterministic wind speed forecast. An interactive display (e.g., a combination of an uncertainty chart with a box plot display) may be optimal to convey uncertainty information.Limor Nadav-Greenberg is a graduate student in cognition and perception in the Department of Psychology, University of Washington. She is also an investigator in the MURI Research Project, a multidisciplinary group of investigators who study and aim to improve the integration and visualization of multisource information for mesoscale meteorology. Details on the project are available at http://www.stat.washington.edu/MURI/. Susan L. Joslyn is a senior lecturer in the Department of Psychology, University of Washington. She is a cognitive psychologist whose research focuses on decision making in applied settings and has worked in a wide range of domains, including emergency dispatch, education, air traffic control, and the use of uncertainty information in weather forecasts. She also teaches several undergraduate and graduate-level courses.

How to Assess Visual Communication of Uncertainty? A Systematic Review of Geospatial Uncertainty Visualisation User Studies

“…Examples include colour hue and colour value (a term under which we subsume value, lightness and brightness) (e.g. Aerts et al, 2003;Edwards and Nelson, 2001;Leitner and Buttenfield, 2000;MacEachren et al, 1998;Nadav-Greenberg et al, 2008;Retchless, 2012;Schweizer and Goodchild, 1992;Slocum et al, 2003), transparency (e.g. Drecki, 2002;Newman and Lee, 2004;Slocum et al, 2003;Viard et al, 2011), or colour saturation that is used in a number of studies (e.g.…”

“…A number of studies define groups with different levels of expertise (e.g. novices versus experts) to assess its impact on the results (Evans, 1997;Kubíček and Šašinka, 2011;Nadav-Greenberg et al, 2008;Schweizer and Goodchild, 1992;Slocum et al, 2003). All in all, the type and level of expertise of participants often remain unclear and groups are usually heterogeneous in expertise.…”

“…We identified two kinds of value retrieval tasks that are used. In the first, data and/or uncertainty values have to be retrieved separately (Aerts et al, 2003;Alberti, 2013;Kubíček and Šašinka, 2011;MacEachren et al, 1998;Nadav-Greenberg et al, 2008;Wittenbrink et al, 1996). In the second, retrieval of both data values and uncertainty happens simultaneously (Blenkinsop et al, 2000;Drecki, 2002;Kolbeinsson, 2013;Kubíček and Šašinka, 2011).…”

“…For instance, in the study conducted by Sanyal et al (2009), search tasks for the lowest and highest uncertainty values resulted in different user performance (although using the same data). From their study about weather forecasting uncertainty, Nadav-Greenberg et al (2008) suggest that 'it is extremely important that designers of such displays consider both user and task demands because the usability and usefulness of uncertainty information depends on these factors' (Nadav-Greenberg et al, 2008, p. 44). In their study, a box plot was successful for precise information whereas the colour-coded maps worked better for relative comparisons.…”

Towards Improved Understanding of the Applicability of Uncertainty Forecasts in the Electric Power Industry

“…Interactive charts permit the user to explore the data and extract information relevant to their specific application. Furthermore, the use of colors has shown to have a stronger impact on memory than abstract information [112]. However, visualizations can be misinterpreted and there are situations in which they may lead the end-user to false conclusions [113].…”