The Benefits and Challenges of Predictive Interval Forecasts and Verification Graphics for End Users

Abstract

Two behavioral experiments tested the use of predictive interval forecasts and verification graphics by nonexpert end users. Most participants were able to use a simple key to understand a predictive interval graphic, showing a bracket to indicate the upper and lower boundary values of the 80% predictive interval for temperature. In the context of a freeze warning task, the predictive interval forecast narrowed user expectations and alerted participants to the possibility of colder temperatures. As a result, participants using predictive intervals took precautionary action more often than did a control group using deterministic forecasts. Moreover, participants easily understood both deterministic and predictive interval verification graphics, based on simple keys, employing them to correctly identify better performing forecast periods. Importantly, participants with the predictive interval were more likely than those with the deterministic forecast to say they would use that forecast type in the future, demonstrating increased trust. Verification graphics also increased trust in both predictive interval and deterministic forecasts when the effects were isolated from familiarity in the second study. These results suggest that forecasts that include an uncertainty estimate might maintain user trust even when the single-value forecast fails to verify, an effect that may be enhanced by explicit verification data.

Entirely possible overruns: how people think and talk about probabilistic cost estimates

“…Graphical displays of variability and probability have in other studies been shown to improve probabilistic judgments (e.g., in the area of health risks) and bring them more in line with formal requirements (Burkell, 2004;Joslyn et al, 2013;Lipkus and Hollands, 1999)…”

Progress and challenges in forecast verification

“…But will enough people be able to understand the verification? Despite the fact that this information is both abstract and complex, there is some preliminary evidence suggesting that non-experts can understand simple verification graphics for both deterministic and probabilistic forecasts (Joslyn et al ., 2013). With no explicit training, using only a simple key, college undergraduates were readily able to identify forecasts that performed better over various forecast periods.…”

Probabilistic Interval Forecasts: An Individual Differences Approach to Understanding Forecast Communication

“…In theory, a range of values and the associated probability provide useful uncertainty information to both. Indeed, research with college student participants suggests that predictive intervals benefit users in just that way, allowing them to better distinguish, than do point estimates, situations in which to take precautionary action [12,13]. In these studies, some participants based their decisions on a forecast that included the upper and lower bound temperatures for an 80% predictive interval as well as a median value described at the "most likely" temperature.…”