Robust inference under time‐varying volatility: A real‐time evaluation of professional forecasters

Abstract

Summary In many forecast evaluation applications, standard tests as well as tests allowing for time‐variation in relative forecast ability build on heteroskedasticity‐and‐autocorrelation consistent (HAC) covariance estimators. Yet, the finite‐sample performance of these asymptotics is often poor. “Fixed‐ b$$ b $$” asymptotics, used to account for long‐run variance estimation, improve finite‐sample performance under homoskedasticity, but lose asymptotic pivotality under time‐varying volatility. Moreover, loss of pivotality due to time‐varying volatility is found in the standard HAC framework in certain cases as well. We prove a wild bootstrap implementation to restore asymptotically pivotal inference for the above and new CUSUM‐ and Cramér‐von Mises‐based tests in a fairly general setup, allowing for estimation uncertainty from either a rolling window or a recursive approach when fixed‐ b$$ b $$ asymptotics are adopted to achieve good finite‐sample performance. We then investigate the (time‐varying) performance of professional forecasters relative to naive no‐change and model‐based predictions in real‐time. We exploit the Survey of Professional Forecasters (SPF) database and analyze nowcasts and forecasts at different horizons for output and inflation. We find that not accounting for time‐varying volatility seriously affects outcomes of tests for equal forecast ability: wild bootstrap inference typically yields convincing evidence for advantages of the SPF, while tests using non‐robust critical values provide remarkably less. Moreover, we find significant evidence for time‐variation of relative forecast ability, the advantages of the SPF weakening considerably after the “Great Moderation.”

Agree to disagree? Predictions of U.S. nonfarm payroll changes between 2008 and 2020 and the impact of the COVID19 labor shock

“…applies D'Agostino et al (2012 methodology on SPF histogram and point forecasts of GDP growth rates and the deflator and finds differences in forecasting ability of the SPF participants. However,Demetrescu et al (2021) raise the issue of a time-varying effect in forecasting ability and by not accounting for this phenomenon, tests for forecasting ability might be biased.A relatively recent strand of literature focuses on the prediction quality of exogenous factors, in particular those linked to an individuals' employment and economic situation Vosen & Schmidt (2011). use different approaches to forecasting private consumption and finds that models including categorized Google search volume outperform survey forecasts.…”