A nonparametric approach to identifying a subset of forecasters that outperforms the simple average

Abstract

Empirical studies in the forecast combination literature have shown that it is notoriously di cult to improve upon the simple average despite the availability of optimal combination weights. In particular, historical performance-based combination approaches do not select forecasters that improve upon the simple average going forward. This paper shows that this is due to the high correlation among forecasters, which only by chance causes some individuals to have lower root mean squared errors (RMSE) than the simple average. We introduce a new nonparametric approach to eliminate forecasters who perform well based purely on chance as well as poor performers. This leaves a subset of forecasters with better performance in subsequent periods. It improves upon the simple average in the SPF for bond yields where some forecasters may be more likely to have specialized knowledge. ⇤ We would like to thank Neil Ericsson, Tatevik Sekhposyan, Herman Stekler and Benjamin Williams for their valuable comments and support. We would also like to thank participants in the Federal Forecasters Conference, the Georgetown Center for Economic Research (GCER) conference, and the GWU SAGE seminar series.

Will the last be the first? Ranking German macroeconomic forecasters based on different criteria

“…Additionally, we follow the approach suggested by Bürgi and Sinclair (2017). They calculate for each institution a dummy variable "that takes value 1 in a given period if that forecaster has a lower squared error in that period than the simple average and 0 otherwise."…”

“…Finally, we adopt the Bürgi and Sinclair (2017) approach outlined above to select successful forecasters for a given year. To this end, we refer to the last five years and demand that an institution should have been better than the average at least half of the time, i.e., the percentage threshold in our case is 50 %.…”

What does forecaster disagreement tell us about the state of the economy?

“…Disagreement however will become substantially larger when recessions are anticipated by at least some forecasters than in normal periods when forecasters tend to bunch together (e.g. see Bürgi and Sinclair (2017)).…”

Robustness of Forecast Combination in Unstable Environment: A Monte Carlo Study of Advanced Algorithms

“…We find that different combination algorithms excel in different kinds of unstable environments: Compared with the rest of the methods we examined in the paper, the one proposed in Sancetta (2010) is more robust to breaks in performances due to idiosyncratic errors; the AFTER algorithms are more robust to unpredictable and sudden aggregate shocks; and the approach in Bürgi & Sinclair (2017) is quicker in adapting to changes in individuals'…”

“…The results, therefore, are closer to those based on equal weights. In addition, we examine the performance of the nonparametric approach proposed in Bürgi & Sinclair (2017), where the authors suggested that only the forecasts with a proven track record of outperforming simple averaging should be combined.…”