Internal Consistency of Survey Respondents' Forecasts: Evidence Based on the Survey of Professional Forecasters*

Abstract

We ask whether the di¤erent types of forecasts made by individual survey respondents are mutually consistent, using the SPF survey data. We compare the point forecasts and central tendencies of probability distributions matched by individual respondent, and compare the forecast probabilities of declines in output with the probabilities implied by the probability distributions. When the expected associations between these di¤erent types of forecasts do not hold for some idividuals, we consider whether the discrepancies we observe are consistent with rational behaviour by agents with asymmetric loss functions.Journal of Economic Literature classi…cation: C53, E32, E37

Explanations of the inconsistencies in survey respondents’ forecasts

“…Recently, a number of authors have sought to assess forecaster rationality in terms of the internal consistency of the di¤erent types of forecasts simultaneously made by individual forecasters. The key papers are Engelberg, Manski and Williams (2007), who compare the point forecasts and histograms of the respondents to the US Survey of Professional Forecasters, and Clements (2008), who in addition assesses the evidence for consistency of the SPF respondents'histograms and forecast probabilities of declines in real output growth. Inconsistencies are found, and are generally in the direction of the point and probability forecasts indicating a rosier outlook than the histograms: the point forecasts of output growth and in ‡ation are higher and lower, respectively, than implied by the histogram forecasts, 1 and the histogram probabilities of declines in output tend to overstate the directly-reported probabilities that respondents assign to such an event.…”

“…Engelberg et al (2007) consider whether the tendency of survey respondents to round the probabilities of their histogram forecasts might be responsible, but …nd that their results are qualitatively unchanged if they allow for rounding. Clements (2008) considers the possibility that respondents' point forecast loss functions are asymmetric (whilst the reported histograms accurately re ‡ect the individuals' true beliefs). Then, the forecaster may rationally report as a point forecast a quantile of their probability distribution other than the central tendency (see, e.g., Elliott, Komunjer and Timmermann (2005a) and Patton and Timmermann (2007)).…”

“…Then, the forecaster may rationally report as a point forecast a quantile of their probability distribution other than the central tendency (see, e.g., Elliott, Komunjer and Timmermann (2005a) and Patton and Timmermann (2007)). As individuals may have loss functions with di¤erent degrees of asymmetry, Clements (2008) tests for rationality allowing for asymmetric loss separately for each individual, based on whether the di¤erence between the histogram mean and point forecasts varies systematically with variables in the information set (other than the conditional variance or standard deviation). We will revisit this issue using panel regressions, as we …nd that forecast uncertainty appears to play an important role, as would be predicted by the asymmetric loss explanation: forecast uncertainty will drive a wedge between the optimal point forecast and the conditional mean.…”

Probability distributions or point predictions? Survey forecasts of US output growth and inflation

“…We suppose they do not use fully-integrated forecasting systems that produce forecast distributions and point forecasts in a mutually consistent fashion, otherwise we ought not observe the inconsistencies documented by Engelberg et al (2009) and Clements (2009Clements ( , 2010.…”

“…For the US Survey of Professional Forecasters (SPF), Engelberg, Manski and Williams (2009) and Clements (2009Clements ( , 2010 examine the relationship between the subjective probability distributions of the individual respondents, which are reported as histograms, and the respondents'point forecasts, for both real output growth and in ‡ation. They conclude that for the majority of cases the two match, but that when they are inconsistent the point forecasts of output growth and in ‡ation tend to suggests a rosier outlook: the output growth and in ‡ation point forecasts are higher and lower, respectively, than measures of central tendency derived from the subjective probability distributions reported as histogram forecasts.…”