Confidence Bands for ROC Curves With Serially Dependent Data

Abstract

We propose serial correlation robust asymptotic confidence bands for the receiver operating characteristic (ROC) curves estimated by quasi-maximum likelihood in the binormal model. Our simulation experiments confirm that this new method performs fairly well in finite samples. The conventional procedure is found to be markedly undersized in terms of yielding empirical coverage probabilities lower than the nominal level, especially when the serial correlation is strong. We evaluate the three-quarter-ahead probability forecasts for real GDP declines from the Survey of Professional Forecasters, and find that one would draw a misleading conclusion about forecasting skill if serial correlation is ignored. JEL Classifications: C01, C12, C13, C53Key words: Receiver operating characteristic curve, Classification, Binormal model, Quasi-maximum likelihood, Survey of Professional Forecasters, Serial correlation. * Corresponding author. Tel.: +1 518 442 4758. E-mail address: klahiri@albany.edu. † We thank Zhongwen Liang for carefully reading an earlier version of the paper and making valuable comments and suggestions.

Bubbles and Crises: The Role of House Prices and Credit

“…As explained in Hsieh and Turnbull () and Berge and Jordà (), AUC follows an asymptotic normal distribution under the null, which also means that a standard Wald type statistic may be used to statistically test the null of no difference in AUC between competing models (Pepe et al , ; Janes et al , ). However, as recently pointed out by Lahiri and Yang (), standard errors for ROC curves and AUC calculated under the i.i.d. assumption may lead to a downward bias if the data are serially correlated, i.e.…”

“…They develop alternative ways of calculating confidence bands that are robust to serial correlation. A weakness of the approach in Lahiri and Yang () is that the confidence bands are derived under the assumption that the classification variable (crisis probability) follows a normal distribution. That said, while we calculate the standard errors under the i.i.d.…”

Eliminating Lining Seams in Tunnel Concrete Crack Images via Line Segments’ Translation and Expansion

“…In this paper, the classification accuracy evaluation method based on confusion matrix [24], [25] is used to conduct the quantitative analysis of the experiment. Table 3 shows the terms of the confusion matrix.…”

Prediction in a Generalized Spatial Panel Data Model with Serial Correlation

“…Extensions of this BLUP should be applied to dynamic spatial panel models (see Baltagi et al , ), and to panel data models with a spatial lag, as well as higher‐order autoregressive and moving average processes (see Baltagi and Liu, ,b). Furthermore, applied researchers may be interested in confidence intervals for serially dependent data (see Lahiri and Yang (), for an example). One might be interested in obtaining confidence intervals of .…”