Backtesting Value-at-Risk and Expected Shortfall in the Presence of Estimation Error

Abstract

We investigate the effect of estimation error on backtests of (multi-period) expected shortfall (ES) forecasts. These backtests are based on first order conditions of a recently introduced family of jointly consistent loss functions for Value-at-Risk (VaR) and ES. We provide explicit expressions for the additional terms in the asymptotic covariance matrix that result from estimation error, and propose robust tests that account for it. Monte Carlo experiments show that the tests that ignore these terms suffer from size distortions, which are more pronounced for higher ratios of outof-sample to in-sample observations. Robust versions of the backtests perform well, although this also depends on the choice of conditioning variables. In an application to VaR and ES forecasts for daily FTSE 100 index returns as generated by AR-GARCH, AR-GJR-GARCH, and AR-HEAVY models, we find that estimation error substantially impacts the outcome of the backtests.

Marginals Versus Copulas: Which Account For More Model Risk In Multivariate Risk Forecasting?

“…Instead, show that ES and VaR are jointly elicitable, see also Acerbi and Székely (2014). There is a growing body of literature building on this result, see, e.g., ; Nolde and Ziegel (2017) for forecast comparisons and Patton et al (2019); Barendse et al (2019); Bayer and Dimitriadis (2020b) for applications in a regression procedure. As the ES is only jointly elicitable with the VaR, we choose a loss function for the ES that is based on both VaR and ES forecasts as input into the MCS procedure.…”

Encompassing Tests for Value at Risk and Expected Shortfall Multi-Step Forecasts based on Inference on the Boundary

“…Furthermore, the asymptotic theory allows for encompassing tests based on any strictly consistent loss function for the VaR and ES. Incorporating estimation risk into these tests can be obtained by combining our theory with the work of Escanciano and Olmo (2010); Du and Escanciano (2017); Barendse et al (2019), and incorporating model misspecification through combining our theory with the one of Dimitriadis and Schnaitmann (2020). Encompassing tests for different functionals such as e.g., the mean, quantiles, expectiles or probability densities based on link functions which require testing on the boundary (e.g., using convex link functions) can be implemented through adapting our asymptotic theory to semiparametric models for the functional under consideration.…”

Assessing the Performance of the Block Maxima Method in Estimating Market Risk

“…The LRcc test, developed by Christoffersen (1998), jointly examines whether the percentage of exceptions is statistically equal to the expected (𝛼 ̂= 𝛼) and the serial independence of the exception indicator. The likelihood ratio statistic of this test is given by 𝐿𝑅 𝑐𝑐 = 𝐿𝑅 𝑢𝑐 + 𝐿𝑅 𝑖𝑛𝑑 , which is asymptotically distributed as 𝜒 2 (2), and the 𝐿𝑅 𝑖𝑛𝑑 statistic is the likelihood ratio statistic for the hypothesis of serial independence against first-order Markov dependence 7 .…”

“…This test has several drawbacks: (i) the use of the Markov chain allows only the influence of past violations to be measured and does not allow for the influence of other 7 The LRind statistic is and has an asymptotic distribution. The likelihood function under the alternative hypothesis is , where Nij denotes the number of observations in state j after having been in state i in the previous period, and .…”