What's under the ROC? An Introduction to Receiver Operating Characteristics Curves

Abstract

Highlights· ROC analysis is used to select the optimal cut point to dichotomize a continuous scale. · The usual choice of cut points minimizes the overall number of false positive and false negative errors. · The cut point may be shifted if the cost of false positives is higher than that of false negatives, or vice versa. · The accuracy of ROC analysis depends on the quality of the gold standard, which is usually far from perfect in psychiatry. · Changing the purpose of the test (for example, from diagnosis to screening) requires a shift in cut points. · A cut point that is ideal for one group may be less than ideal for another.

Anthropogenic activities influence the abandonment of Bearded Vulture (Gypaetus barbatus) territories in southern Africa

“…Our overall model had an AUC value of 0.69, and the models for South Africa and Lesotho had AUC values of 0.70 and 0.68, respectively, thus classifying them as poor to moderate fit or low to medium accuracy (Streiner and Cairney 2007). …”

“…The area under the curve (AUC) is a good numerical index (Hanley and (Streiner and Cairney 2007); therefore, the higher the AUC value, the better the fit of the model. Low AUC values, however, do not necessarily indicate a poor model; rather they suggest that factors other than the predictor variables may also be influencing the response variable (Nemes and Hartel 2010).…”

The quality of spine surgery from the patient’s perspective: Part 2. Minimal clinically important difference for improvement and deterioration as measured with the Core Outcome Measures Index

“…The corresponding value (AUC) for predicting deterioration was 0.89. The accuracy (or discriminatory ability) of tests with AUCs 0.50-0.70 is generally considered low, between 0.70 and 0.90, moderate, and over 0.90, high [23]. The AUCs for the COMI were comparable to or even higher than those previously reported in the literature for back-specific instruments such as the Oswestry Disability Index (AUC 0.82-0.85) [6,13,16], the RolandMorris Disability Questionnaire (AUC 0.84) [16], or the pain intensity visual analog scale (AUC 0.88) [16] in patients undergoing spine surgery; they were also slightly higher than the AUC reported for the COMI during its initial validation studies in surgical patients (AUC 0.82) [15].…”

Subthreshold depression based on functional impairment better defined by symptom severity than by number of DSM-IV symptoms

“…We identified the range of WHODAS II scores which generated the highest sum of sensitivity and specificity. This criterion minimizes the number of false positives and false negatives, thus minimizing chances that respondents within the given WHODAS II range would falsely be placed above or below the identified cut-off point (Streiner and Cairney, 2007).…”