The Wisdom of Competitive Crowds

Abstract

When several individuals are asked to forecast an uncertain quantity, they often face implicit or explicit incentives to be the most accurate. Despite the desire to elicit honest forecasts, such competition induces forecasters to report strategically and nontruthfully. The question we address is whether the competitive crowd's forecast (the average of strategic forecasts) is more accurate than the truthful crowd's forecast (the average of truthful forecasts from the same forecasters). We analyze a forecasting competition in which a prize is awarded to the forecaster whose point forecast is closest to the actual outcome. Before reporting a forecast, we assume each forecaster receives two signals: one common and one private. These signals represent the forecasters' past shared and personal experiences relevant for forecasting the uncertain quantity of interest. In a set of equilibrium results, we characterize the nature of the strategic forecasts in this game. As the correlation among the forecasters' private signals increases, the forecasters switch from using a pure to a mixed strategy. In both cases, forecasters exaggerate their private information and thereby make the competitive crowd's forecast more accurate than the truthful crowd's forecast.

Incentives for self-extremized expert judgments to alleviate the shared-information problem.

“…Lichtendahl Jr. et al ( 2013) establish a limiting equilibrium in a normal model where winner-take-all contests elicit self-extremized expert predictions in large crowds of experts. Note that for K = N and N → ∞, the optimal weight on private signals is one for any l > 0 and we have α 2 → 1 in the equilibrium above . Lichtendahl Jr. et al (2013) also show that, depending on the parameters, the limiting weight on the private signal is one either in a symmetric pure strategy equilibrium or in a mixed strategy equilibrium where experts provide a noisy report of their private signal only.…”

“…Study 2 also implemented a winner-take-all contest as an alternative incentive-based solution to elicit self-extremized expert judgments. Lichtendahl Jr. et al (2013) derived the limiting equilibria in a winner-take-all contest where experts self-extremize. The resulting average forecast is more accurate than the average of nonextremized forecasts.…”

“…Note that for K = N and N → ∞, the optimal weight on private signals is one for any ℓ > 0 and we have α2→1 in the equilibrium above. Lichtendahl Jr. et al (2013) also show that, depending on the parameters, the limiting weight on the private signal is one either in a symmetric pure strategy equilibrium or in a mixed strategy equilibrium where experts provide a noisy report of their private signal only. These equilibria achieve optimal weighting of signals for N → ∞.…”

“…Lichtendahl Jr. et al (2013) establish a limiting equilibrium in a normal model where winner-take-all contests elicit self-extremized expert predictions in large crowds of experts. Note that for K = N and N → ∞, the optimal weight on private signals is one for any ℓ > 0 and we have α2→1 in the equilibrium above.…”

A survey of human judgement and quantitative forecasting methods

“…Ottaviani & Sørensen [28] and Lichtendahl & Winkler [29] have shown that forecasters who compete to be the most accurate have an incentive to not report their true opinion. Lichtendahl et al [30] have shown that an advice seeker benefits from setting up an incentive scheme that rewards strategic behaviour. The rationale is that the variance of the quantity predicted by the experts is greater under a competitive scheme.…”