Subjective confidence in forecasts

Abstract

Forecasts have little value to decision makers unless it is known how much confidence to place in them. Those expressions of confidence have, in turn, little value unless forecasters are able to assess the limits of their own knowledge accurately Previous research has shown very robust patterns in the judgements of individuals who have not received special training in confidence assessment: Knowledge generally increases as confidence increases. However, it increases too swiftly, with a doubling of confidence being associated with perhaps a 50 per cent increase in knowledge. With all but the easiest of tasks, people tend to be overconfident regarding how much they know These results have typically been derived from studies of judgements of general knowledge. The present study found that they also pertained to confidence in forecasts. Indeed, the confidence‐knowledge curves observed here were strikingly similar to those observed previously. The only deviation was the discovery that a substantial minority of judges never expressed complete confidence in any of their forecasts. These individuals also proved to be better assessors of the extent of their own knowledge Apparently confidence in forecasts is determined by processes similar to those that determine confidence in general knowledge. Decision makers can use forecasters assessments in a relative sense, in order to predict when they are more and less likely to be correct. However, they should be hesitant to take confidence assessments literally. Someone is more likely to be right when he or she is ‘certain’than when he or she is ‘fairly confident’; but there is no guarantee that the certain forecast will come true.

Confidence interval estimation tasks and the economics of overconfidence

“…However, it is an inevitable problem that knowledge questions given to professionals in their areas of expertise are likely to be interpreted as tests of primary knowledge, and thus ability. Previous research has indicated that experts are, in most cases, equally prone to biased judgments (Fischhoff and MacGregor, 1982). But such an assertion overlooks the fact that the job of experts is typically to forecast the value of a realized stochastic variable (e.g.…”

Human Factors in High-Altitude Mountaineering

“…The overconfidence bias is one in which people often tend to be more confident than they have a right to be in the accuracy of their forecast or diagnosis (Bornstein & Zickafoos, 1999;Fischoff & MacGregor, 1982;Kahneman, 2011;Wickens et al, 2013). This is often as true of experts as of novices (Kahneman, 2011;Taleb, 2007).…”

The Ombudsman: Research on Forecasting: A Quarter-Century Review, 1960–1984

“…I have replicated this study many times and obtained similar results. As people gain expertise in an area, their tendency toward overconfidence increases [Fischhoff and MacGregor 1982]. 1 suggest that you ignore confidence claims by individuals, except where feedback about outcomes is readily available and easy to summarize.…”