Robustness of Forecast Combination in Unstable Environment: A Monte Carlo Study of Advanced Algorithms

Abstract

In this paper, we study the behavior and effectiveness of several recently developed forecast combination algorithms in simulated unstable environments, where the performances of individual forecasters are cross-sectionally heterogeneous and dynamically evolving. Our results clearly reveal how different algorithms respond to structural instabilities of different origin, frequency, and magnitude. Accordingly, we propose an improved forecast combination procedure and demonstrate its effectiveness in a real-time forecast combination exercise using the U.S. Survey of Professional Forecasters.

Predicting U.S. Business Cycle Turning Points Using Real-Time Diffusion Indexes Based on a Large Data Set

“…There is a practical limitation to the analysis of the probit forecasts due to the relatively short time series at our disposal. Our real-time data set contains only the 2007-2009recession. Two additional recessions (1991and 2001 happened during the longer sample period for which the pseudo-real-time data are available.…”

“…Zhao (2016) shows that estimating even a set of five weights using a short time series in real-time may produce combined forecasts that are inferior to simply averaging the individual forecasts. This is especially true when the target variable or individual's forecasts are subject to structural breaks.…”

Predicting U.S. Business Cycle Turning Points Using Real-Time Diffusion Indexes Based on a Large Data Set

“…18 Zhao (2016) shows that estimating even a set of five weights using a short time series in real-time may produce combined forecasts that are inferior to simply averaging the individual forecasts. This is especially true when the target variable or individual's forecasts are subject to structural breaks.…”