Testing for Differences in Path Forecast Accuracy: Forecast-Error Dynamics Matter

Martinez, Andrew B.

doi:10.26509/frbc-wp-201717

Cited by 5 publications

(2 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the context of model comparison, Capistrán (2006) introduces an unweighted version of the average SPA test. Subsequent research by Martinez (2017) provides a generalization of the unweighted average SPA test in a GFESM context (Clements and Hendry 1993), explicitly allowing for differences in covariance dynamics of the various models, while we target the loss-differential directly as a primitive. Finally, the literature on vector forecasts, concerning multiple variables rather than multiple horizons, faces the similar problem of forecast comparison in the presence of correlated forecast errors (e.g., Clements and Hendry 1993;Komunjer and Owyang 2012).…”

Section: Introductionmentioning

confidence: 99%

Multi-Horizon Forecast Comparison

Quaedvlieg

2017

SSRN Journal

View full text Add to dashboard Cite

We introduce tests for multi-horizon superior predictive ability (SPA). Rather than comparing forecasts of different models at multiple horizons individually, we propose to jointly consider all horizons of a forecast path. We define the concepts of uniform and average SPA. The former entails superior performance at each individual horizon, while the latter allows inferior performance at some horizons to be compensated by others. The article illustrates how the tests lead to more coherent conclusions, and how they are better able to differentiate between models than the single-horizon tests. We provide an extension of the previously introduced model confidence set to allow for multi-horizon comparison of more than two models. Simulations demonstrate appropriate size and high power. An illustration of the tests on a large set of macroeconomic variables demonstrates the empirical benefits of multi-horizon comparison.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-Horizon Forecast Comparison

Quaedvlieg

2017

SSRN Journal

View full text Add to dashboard Cite

show abstract

“…A number of recent studies propose methods to compare the relative accuracy of path forecasts (Capistrán, 2006; Patton & Timmermann, 2012; Martinez, 2017). In our analysis, we use the tests of multihorizon superior predictive ability proposed by Quaedvlieg (2021) that jointly consider all horizons along the entire projection path.…”

Section: Methodsmentioning

confidence: 99%

The accuracy and informativeness of agricultural baselines

Bora

Katchova

Kuethe

2022

American J Agri Economics

View full text Add to dashboard Cite

Agricultural baselines facilitate policy and investment decisions by governments and market participants by providing long‐term projections about the farm sector. Despite their importance in shaping agricultural policy, the agricultural baselines have not been rigorously evaluated. This study evaluates the accuracy and informativeness of two widely used baselines for the US farm sector published by the United States Department of Agriculture (USDA) and the Food and Agricultural Policy Research Institute (FAPRI) in three steps. First, we examine the average percent errors of the projections and perform tests of bias. Second, we use a testing framework based on the encompassing principle to test the predictive content of the projections for each horizon, determining the longest informative projection horizon. Third, we compare the USDA and FAPRI baseline projections using a multi‐horizon framework that considers all projection horizons jointly. We find that prediction error and bias increase with the horizon's length. The predictive content of the baselines projections for most variables diminishes after 4–5 years. The multi‐horizon comparison suggests that neither USDA nor FAPRI projections have uniform or average superior predictive ability over the other for most variables. Our findings are useful for the agencies producing these baselines and for the policymakers, agricultural businesses, and other stakeholders who use them. The study contributes to the recent literature on long‐term agricultural projections and establishes the groundwork for future research inquiries.

show abstract

Evaluating Government Budget Forecasts

Ericsson

Martinez

2019

Palgrave Studies in Public Debt, Spending, and Revenue

View full text Add to dashboard Cite

Testing for Differences in Path Forecast Accuracy: Forecast-Error Dynamics Matter

Cited by 5 publications

References 40 publications

Multi-Horizon Forecast Comparison

Multi-Horizon Forecast Comparison

The accuracy and informativeness of agricultural baselines

Evaluating Government Budget Forecasts

Contact Info

Product

Resources

About