Accuracy and cut‐off point selection in three‐class classification problems using a generalization of the Youden index

Abstract

SummaryWe study properties of the index J 3 , defined as the accuracy, or the maximum correct classification, for a given three-class classification problem. Specifically, using J 3 one can assess the discrimination between the three distributions and obtain an optimal pair of cut-off points c 1 < c 2 in the sense that the sum of the correct classification proportions will be maximized. It also serves as the generalization of the Youden index in three-class problems. Parametric and nonparametric approaches for estimation and testing are considered and methods are applied to data from an MRS study on HIV patients.

Validating a Pragmatic Approach to Cognitive Screening in Stroke Prevention Clinics Using the Montreal Cognitive Assessment

“…Radiologists have long recognized this approach and have used receiver operator characteristic (ROC) curves to facilitate comparison across the range of diagnostic certainty (eg, definitely normal, probably normal, equivocal, probably abnormal, and almost certainly abnormal). 20,21 Statistical approaches to 3-class classification problems have been developed 22,23 and expanded beyond to multiple class classification problems. 24 This approach has been used to support clinical decision making, for example, in screening patients presenting with chest pain, 25 or in differentiating normal aging from CI and dementia.…”

“…The same concept applies to the methodology of classifying patients when using 2 cut points as with that of a single, optimized cut point. 23,25,26 When a single cut point was chosen, those who scored below the single cut point generated for both high specificity and specificity formed the high-risk group, and those who scored above the cut point formed the low-risk group. When 2 cut points were chosen, those who scored below the cut point generated for high specificity formed the high-risk group and those who scored above the cut point generated for high sensitivity formed the low-risk group.…”

Three-group ROC predictive analysis for ordinal outcomes

“…One approach is Youden's index (Youden, 1950), which for three-group (continuous) diagnostic tests was introduced by Nakas et al (2010). Similarly we can define Youden's index for ordinal three-group diagnostic tests as…”

ThresholdROC: Optimum Threshold Estimation Tools for Continuous Diagnostic Tests in R

“…However, the diagnostic problem can also include more than two classification states, such as "non-diseased", "mild condition" or "severe condition". The ability of the diagnostic marker to discriminate between states is usually evaluated with the area under the receiver operating characteristic (ROC) curve in the two-state setting (Metz 1978;Pepe 2003) and the volume under the surface (VUS) for the three-state setting (Nakas, Alonzo, and Yiannoutsos 2010).…”

“…To estimate the threshold that optimally discriminates between states, standard methods consist of choosing a threshold for a desired false positive/negative rate to be achieved or, more formally, by maximizing the Youden index, which is the sum, diminished by unity, of the two fractions showing the proportions correctly classified (Youden 1950;Nakas et al 2010). Another method based on defining an overall cost function, which includes correct and incorrect classification rates and the relevant weights associated with each decision, thus allowing disease prevalence to be also considered, was proposed (Metz 1978;Pepe 2003) and further developed (Jund, Rabilloud, Wallon, and Ecochard 2005;Skaltsa, Jover, and Carrasco 2010;Skaltsa, Jover, Fuster, and Carrasco 2012).…”