How Likely is That Chance of Thunderstorms? A Study of How National Weather Service Forecast Offices Use Words of Estimative Probability and What They Mean to the Public

Abstract

One of the challenges when communicating forecast information to the public is properly contextualizing uncertainty. No forecast is ever certain, as no meteorological phenomenon is guaranteed to occur. As such, the uncertainty in forecast information should be communicated in a way that makes sense to end users. Previous studies of the communication of probabilistic information suggest that, although the general public are more apt to communicate uncertainty with words of estimative probability (WEPs), they prefer to receive uncertainty information numerically. Other work has suggested that a combination of numbers and WEPs is the best method for communicating probability, but fewer research studies have assessed the communication and interpretation of probabilistic meteorological information. In this study, we code 8900 tweets from the National Weather Service Weather Forecast Offices (WFOs) and analyze them to find how forecasters communicate probabilistic forecast information to the public via Twitter. This analysis reveals that WFO messaging is dominated by WEPs, with few numerical descriptions of probability. These WEPs are generally vague, unqualified notions of probability that may impede the public’s ability to interpret the information that forecasters are trying to communicate. Based on this analysis, two publicly fielded surveys also are analyzed in order to understand how participants tend to interpret the most common qualified and unqualified WEPs that WFOs used on Twitter. Though participants generally interpret qualified WEPs more consistently than unqualified WEPs, both categories featured a wide range of interpretations that suggest both types of WEPs are vaguely defined for the general public.

Facilitating Sender-Receiver Agreement in Communicated Probabilities: Is it Best to Use Words, Numbers or Both?

“…Many organizations and professional groupings that produce expert judgments prefer to express uncertainties with verbal probabilities such as "likely" or "unlikely" rather than with precise numeric probabilities such as "70% chance" or imprecise numeric ranges such as "60% to 80% chance" (e.g., Ho et al, 2015). For instance, in a recent study of National Weather Service tweets, 99.9% of probabilistic forecasts were made using verbal probability expressions (Lenhardt et al, 2020). Accountants also tend to prefer using verbal probabilities, despite the quantitative basis of the profession (Kolesnika et al, 2019).…”

Use and perception of weather forecast information across Europe

“…Although NMHSs provide verification of probabilistic forecasts or organize training sessions and post‐event reviews (see Section 3.2), this usually does not take place on regular basis. Brief explanation of probability or intensity of the forecasted event in the form of text, which is sometimes provided by meteorological offices, can be helpful but allows a wide range of interpretation (Abraham et al, 2015; Lenhardt et al, 2020).…”

Exploring Community Differences in Tornado Warning Reception, Comprehension, and Response across the United States

“…In addition to this set of composite scales, WxDash provides information on risk perceptions across a variety of hazards [see Allan et al (2019) for more information], data on tornado warning and extreme weather information sources, and information on how people interpret verbal probability phrases such as "high chance" or "low probability" in severe weather forecasts [ Fig. 4d; see Lenhardt et al (2019) for more information]. In addition to interacting with these data, users are able to download a database of the estimates we produce, the raw survey data we use to calculate them, and the code necessary to reproduce the calculations.…”