Cultivating credibility with probability words and numbers

Abstract

Recent research suggests that communicating probabilities numerically rather than verbally benefits forecasters’ credibility. In two experiments, we tested the reproducibility of this communication-format effect. The effect was replicated under comparable conditions (low probability, inaccurate forecasts), but it was reversed for low-probability accurate forecasts and eliminated for high-probability forecasts. Experiment 2 further showed that verbal probabilities convey implicit recommendations more clearly than probability information, whereas numeric probabilities do the opposite. Descriptively, the findings indicate that the effect of probability words versus numbers on credibility depends on how these formats convey directionality differently, how directionality implies recommendations even when none are explicitly given, and how such recommendations correspond with outcomes. Prescriptively, we propose that experts distinguish forecasts from advice, using numeric probabilities for the former and well-reasoned arguments for the latter.

Arithmetic computation with probability words and numbers

“…Rather, our findings suggest that when communicated probabilities serve as inputs for judgments or decisions that require mathematical computation of those values, they should be communicated as numeric point probabilities. As noted earlier, point probabilities are rated as conveying probability information more clearly than verbal probabilities (Collins & Mandel, 2019). Using point probabilities would also enable more granular assessments to be made and communicated to others, and this has been shown to yield substantial accuracy gains in forecasting (Friedman, Baker, Mellers, Tetlock, & Zeckhauser, 2018).…”

Arithmetic computation with probability words and numbers

“…Rather, our findings suggest that when communicated probabilities serve as inputs for judgments or decisions that require mathematical computation of those values, they should be communicated as numeric probabilities. As noted earlier, numeric probabilities are rated as conveying probability information more clearly than verbal probabilities (Collins & Mandel, 2019). Using numeric probabilities would also enable more granular assessments to be made and communicated to others, and this has been shown to yield substantial accuracy gains in forecasting (Friedman, Baker, Mellers, Tetlock, & Zeckhauser, 2018).…”

“…Such variability and context dependency implies that even if probability words can be operated on arithmetically, the same probability information may be interpreted differently by different judges and even by the same judges across time. People also rate verbal probabilities as conveying probability levels less clearly than numeric probabilities (Collins & Mandel, 2019), a tendency that may have deleterious consequences for receivers' subsequent computations. In fact, even though numeric probabilities may be more susceptible to conservatism in belief updating, processing numeric probabilities has been shown to yield more accurate and reliable judgments than verbal probabilities (Budescu, Weinberg, & Wallsten, 1988;Rapoport, Wallsten, Erev, & Cohen, 1990).…”

Numerically Bounded Linguistic Probability Schemes Are Unlikely to Communicate Uncertainty Effectively

“…For example, terms that convey comparable probability levels can signal optimism (e.g., some chance ) or pessimism (e.g., doubtful ) regarding desirable future events (Teigen & Brun, 1995, 1999). Collins and Mandel (2019) found that individuals perceive linguistic probabilities as communicating probability levels less clearly than do numerical probabilities. They also found that, for low‐probability terms, individuals perceived implicit recommendations more clearly from the verbal probability term than from the numeric probability, even though the communicators provided no explicit recommendations.…”