A new set of improved Value-at-Risk backtests

Abstract

We propose a new set of formal backtests for VaR-forecasts that significantly improve upon existing backtesting procedures. Our new test of unconditional coverage can be used for both directional and non-directional testing and is thus able to test separately whether a VaR-model is too conservative or underestimates the actual risk exposure. Second, we stress the importance of testing the property of independent and identically distributed (i.i.d.) VaRexceedances and propose a simple approach that explicitly tests for the presence of clusters in VaR-violation processes. Results from a simulation study indicate that our tests significantly outperform competing backtests in several distinct settings. In addition, the empirical analysis of a unique data set consisting of asset returns of an asset manager's portfolios underline the usefulness of our new backtests especially in times of market turmoil.Keywords: Value-at-Risk, backtesting, Monte Carlo simulation.JEL Classification: C52, C53, C58. * We thank Harald Rosenthal of RobinNova GmbH for providing the data set for our empirical application and helpful comments on an earlier version of this paper. Judith Lameyer and Janina Mühlnickel provided outstanding research assistance. A New Set of Improved Value-at-Risk Backtests ABSTRACTWe propose a new set of formal backtests for VaR-forecasts that significantly improve upon existing backtesting procedures. Our new test of unconditional coverage can be used for both directional and non-directional testing and is thus able to test separately whether a VaR-model is too conservative or underestimates the actual risk exposure.Second, we stress the importance of testing the property of independent and identically distributed (i.i.d.) VaRexceedances and propose a simple approach that explicitly tests for the presence of clusters in VaR-violation processes.Results from a simulation study indicate that our tests significantly outperform competing backtests in several distinct settings. In addition, the empirical analysis of a unique data set consisting of asset returns of an asset manager's portfolios underline the usefulness of our new backtests especially in times of market turmoil.

How accurate are modern Value-at-Risk estimators derived from extreme value theory?

“…For example, if the error probability is set to be π = 5%, thenp = 7% (p = 3%) implies that our VaR estimator predicts that the calculated VaR will only be exceeded with a probability of 5% but, in fact, it is exceeded more (less) frequently. While a good VaR model would be one that generated ap close to π, a situationp > π, where the true risk is underestimated, is likely to be far more serious for a firm (because risk management activities based on the VaR estimates may lead to insufficient hedges) than having too much capital reserves in the case of p < π indicating an overestimated VaR (see Brooks et al, 2005;Ziggel et al, 2014). 21,22 Table 2 reports the violation rates for the unconditional versions of our EVT-based VaR estimators and, for comparison, also for selected traditional approaches.…”

“…Finally, while the L-moment method now appears to be preferable in terms of MAPE, the Box-Cox method and the Johnson method (i.e., their MPE) again show a tendency to overestimation and can thus be interpreted as the most conservative estimators. Bali et al, 2008;Candelon et al, 2011;Ziggel et al, 2014). In this framework, a sequence of VaR forecasts with specified nominal error probability π is said to be efficient with respect to the information set Ω t−1 if…”

“…A model is disputable (yellow zone) up to the 99.99% quantile and is deemed seriously flawed (red zone) whenever more violations occur (see Kuester et al, 2006;Campbell, 2007). Thus, with the decision rule 'reject the null hypothesis of a valid VaR model whenever the model scores red', the procedure can be interpreted as a significance test, i.e., basically as a one-sided version of the unconditional coverage test (see Ziggel et al, 2014). Not-rejecting the two-sided version directly implies not-rejecting the one-sided version.…”

“…That is, for example, that the nominal error probability 1 − α used in the VaR estimation should equal the frequency of violations (i.e., cases where the actual losses exceed the estimated VaR). The statistical test for this property (the unconditional coverage test), which is based on the binomial distribution, is currently indirectly embodied in the 1996 Market Risk Amendment to the Basel Capital Accord (see Basel Committee on Banking Supervision, 1996b; Ziggel et al, 2014) which requires that commercial banks with significant trading activities set aside capital to cover the market risk exposure by their trading accounts. The market risk capital requirements are to be based on the VaR estimates generated by the banks' own risk management models provided that they pass this test.…”

Does the Assumption on Innovation Process Play an Important Role for Filtered Historical Simulation Model?

“…Statistical accuracy of the models is evaluated by backtests of Kupiec (1995), Christoffersen (1998), Engle and Manganelli (2004) and Sarma et al (2003). Recently, some alternative backtesting methods for VaR forecasts were proposed by Ziggel et al (2014) and Dumitrescu et al (2012). Kupiec (1995) proposed a likelihood ratio (LR) test of unconditional coverage (LR uc ) to evaluate the model accuracy.…”