SummaryThe Youden Index is a frequently used summary measure of the ROC (Receiver Operating Characteristic) curve. It both, measures the effectiveness of a diagnostic marker and enables the selection of an optimal threshold value (cutoff point) for the marker. In this paper we compare several estimation procedures for the Youden Index and its associated cutoff point. These are based on (1) normal assumptions; (2) transformations to normality; (3) the empirical distribution function; (4) kernel smoothing. These are compared in terms of bias and root mean square error in a large variety of scenarios by means of an extensive simulation study. We find that the empirical method which is the most commonly used has the overall worst performance. In the estimation of the Youden Index the kernel is generally the best unless the data can be well transformed to achieve normality whereas in estimation of the optimal threshold value results are more variable.

The area under the receiver operating characteristic curve is frequently used as a measure for the effectiveness of diagnostic markers. In this paper we discuss and compare estimation procedures for this area. These are based on (i) the Mann-Whitney statistic; (ii) kernel smoothing; (iii) normal assumptions; (iv) empirical transformations to normality. These are compared in terms of bias and root mean square error in a large variety of situations by means of an extensive simulation study. Overall we find that transforming to normality usually is to be preferred except for bimodal cases where kernel methods can be effective.

After establishing the utility of a continuous diagnostic marker investigators will typically address the question of determining a cut-off point which will be used for diagnostic purposes in clinical decision making. The most commonly used optimality criterion for cut-off point selection in the context of ROC curve analysis is the maximum of the Youden index. The pair of sensitivity and specificity proportions that correspond to the Youden index-based cut-off point characterize the performance of the diagnostic marker. Confidence intervals for sensitivity and specificity are routinely estimated based on the assumption that sensitivity and specificity are independent binomial proportions as they arise from the independent populations of diseased and healthy subjects, respectively. The Youden index-based cut-off point is estimated from the data and as such the resulting sensitivity and specificity proportions are in fact correlated. This correlation needs to be taken into account in order to calculate confidence intervals that result in the anticipated coverage. In this article we study parametric and non-parametric approaches for the construction of confidence intervals for the pair of sensitivity and specificity proportions that correspond to the Youden index-based optimal cut-off point. These approaches result in the anticipated coverage under different scenarios for the distributions of the healthy and diseased subjects. We find that a parametric approach based on a Box-Cox transformation to normality often works well. For biomarkers following more complex distributions a non-parametric procedure using logspline density estimation can be used.

The area under the receiver operating characteristic curve is the most commonly used measure of the ability of a biomarker to distinguish between two populations. Some markers are subject to substantial measurement error. Under normality assumptions, the authors develop a confidence interval procedure for the area under the receiver operating characteristic curve that adjusts for measurement error. This procedure assumes the availability of data from a reliability study of the biomarker. A simulation study was used to check the validity of the proposed confidence interval. Furthermore, it was shown that not adjusting for measurement error could result in a serious understatement of the effectiveness of the biomarker.

In order to compare the discriminatory effectiveness of two diagnostic markers the equality of the areas under the respective Receiver Operating Characteristic Curves is commonly tested. A non-parametric test based on the Mann-Whitney statistic is generally used. Weiand et al. (1989) present a parametric test based on normal distributional assumptions. We extend this test using the Box-Cox power family of transformations to non-normal situations. These three test procedures are compared in terms of significance level and power by means of a large simulation study. Overall we find that transforming to normality is to be preferred. An example of two pancreatic cancer serum biomarkers is used to illustrate the methodology.