Optimal and robust combination of forecasts via constrained optimization and shrinkage

Abstract

Combining forecasts formed by various models can substantially improve the prediction performances compared to those obtained from the individual models. Standard combination approaches consist in a forecast selection step followed by a weighting scheme. It is not clear, however, which models to include, and how to combine them. This is a central question, having a substantial impact on the quality of the aggregate forecast. We propose a robust method that mitigates estimation uncertainty and implicitly features forecast selection. Our approach relies on constrained optimization with penalty (COP). We take advantage of the equivalence existing between COP and constrained optimization with shrinkage of the prediction errors' covariance matrix (COS) to determine the optimal L2 penalty, thereby making the economoy of an expensive (and potentially harmful) crossvalidation stage. Our method is tested empirically in a simulation exercise and on two applications in economics. The proposed combination schemes outperform the simple average forecast, trimmed simple average forecast and perform at least as well as the best individual model(s) in the considered cases.

Meta-Learning Approaches for Recovery Rate Prediction

“…The latter consists of combining the sample covariance matrix (which is easy to compute, asymptotically unbiased but prone to estimation errors) with an estimator that is misspecified and biased but more robust to estimation errors (Ledoit and Wolf 2004). This approach, initially derived in a portfolio optimization context, was recently transposed in Roccazzella et al (2021) to the forecast combination problem, showing that constrained optimization with shrinkage (COS) of Σ can provide a single-step, fast and robust optimal forecast combination strategy. Here we adapt the COS to act as a linear meta-learner, which combines the first-level learners only on the basis of in-sample information.…”

“…However, several other alternatives are worth exploring. For example, Roccazzella et al (2021) propose a robust and sparse combination method that mitigates estimation uncertainty and implicitly features forecast selection in a single step. This approach has the additional advantage of relying on a closed-form solution for the estimation of the optimal combination strategy.…”

High Dimensional Forecast Combinations Under Latent Structures

“…It is worth mentioning that various shrinkage and machine learning techniques have been applied to the forecasting literature since the pioneer work of Tibshirani (1996). See, e.g., Li and Chen (2014), Conflitti et al (2015), Konzen and Ziegelmann (2016), Stasinakis et al (2016), Bayer (2018), Wilms et al (2018), Kotchoni et al (2019), Coulombe et al (2020), andRoccazzella et al (2020). We complement these works by considering an ℓ 2 -relaxation of the regularized weights estimation problem, which exhibits certain optimality properties when the forecast error VC matrix can be decomposed into the sum of a low-rank matrix and a VC of idiosyncratic shocks.…”

Method for Determining the Optimal Capacity of Energy Storage Systems with a Long-Term Forecast of Power Consumption

“…It is also important to mention that ensemble models have become widespread in recent years, because of combining the advantages of multiple predictive models. The authors of [50] discussed an algorithm for selecting predictive models and combining them using Penalized constrained optimization and shrinkage techniques to predict house prices in Boston. The model consists of 14 predictive algorithms, and the full dataset includes only 507 observations.…”

“…The model consists of 14 predictive algorithms, and the full dataset includes only 507 observations. However, the authors of article [50] also argue that there is a risk of including models in the ensemble, which would worsen the forecast results.…”