Nonparametric Predictive Inference for Accuracy of Ordinal Diagnostic Tests

Abstract

We introduce nonparametric predictive inference (NPI) for accuracy of diagnostic tests with ordinal outcomes, with the inferences based on data for a disease group and a non-disease group. We introduce empirical and NPI lower and upper receiver operating characteristic (ROC) curves and the corresponding areas under the curves, and we prove that these are nested, with the latter equal to the NPI lower and upper probabilities for correctly ordered future observations from the non-disease and disease groups. We discuss the use of the Youden index related to the NPI lower and upper ROC curves in order to determine the optimal cutoff point for the test.

Three-group ROC predictive analysis for ordinal outcomes

“…In this paper we generalize the results in (Elkhafifi and Coolen, 2012) by presenting NPI for three-group ROC surface with ordinal outcomes. In order to use NPI with ordinal data, we use an assumed underlying latent variable representation, with the categories represented by intervals on the real-line, reflecting the known ordering of the categories and enabling application of the assumption A (n) (Coolen et al, 2013;Elkhafifi and Coolen, 2012).…”

“…In NPI, attention is restricted to one or more future observable random quantities, and Hill's assumption A (n) (Hill, 1968) is used to link these random quantities to data, in a way that is closely related to exchangeability (De Finetti, 1974). NPI has been introduced for assessing the accuracy of a classifier's ability to discriminate between two outcomes (or two groups) for binary data (Coolen-Maturi et al, 2012a) and for diagnostic tests with ordinal observations (Elkhafifi and Coolen, 2012) and with real-valued observations (Coolen-Maturi et al, 2012b). Recently, Coolen-Maturi et al (2014) generalized the results in (CoolenMaturi et al, 2012b) by introducing NPI for three-group ROC surface, with real-valued observations, to assess the ability of a diagnostic test to discriminate among three ordered classes or groups.…”

“…In order to use NPI with ordinal data, we use an assumed underlying latent variable representation, with the categories represented by intervals on the real-line, reflecting the known ordering of the categories and enabling application of the assumption A (n) (Coolen et al, 2013;Elkhafifi and Coolen, 2012). The paper is organized as follows.…”

Three-group ROC analysis: A nonparametric predictive approach

“…Attention has been restricted to real-valued data, developing the related NPI theory for ROC surfaces in case of ordinal data is an interesting topic for future research (Elkhafifi and Coolen, 2012;. The concepts and ideas presented can be generalized to classification into more than three categories (Waegeman et al, 2008), but the computation of NPI lower and upper ROC hypersurfaces, in line with Section 4.4, will require numerical optimisation which will be complicated for larger data sets with substantial overlap between observations from different groups.…”

“…Due to the importance of prediction of the accuracy of diagnostic tests for a future patient, NPI provides an attractive alternative approach to the established methods in this field. NPI has recently been introduced for assessing the accuracy of a classifier's ability to discriminate between two groups for binary data (Coolen-Maturi et al, 2012a), ordinal data (Elkhafifi and Coolen, 2012) and real-valued data (Coolen-Maturi et al, 2012b). This paper introduces NPI for three-group ROC analysis for real-valued data.…”