Assessment and comparison of prognostic classification schemes for survival data

Abstract

Prognostic classification schemes have often been used in medical applications, but rarely subjected to a rigorous examination of their adequacy. For survival data, the statistical methodology to assess such schemes consists mainly of a range of ad hoc approaches, and there is an alarming lack of commonly accepted standards in this field. We review these methods and develop measures of inaccuracy which may be calculated in a validation study in order to assess the usefulness of estimated patient‐specific survival probabilities associated with a prognostic classification scheme. These measures are meaningful even when the estimated probabilities are misspecified, and asymptotically they are not affected by random censorship. In addition, they can be used to derive R2‐type measures of explained residual variation. A breast cancer study will serve for illustration throughout the paper. Copyright © 1999 John Wiley & Sons, Ltd.

Hidden Treasures in “Ancient” Microarrays: Gene-Expression Portrays Biology and Potential Resistance Pathways of Major Lung Cancer Subtypes and Normal Tissue

“…Particularly, we measured predictive performances in terms of concordance index (CI) and integrated brier score (IBS) (20). The first metric has an interpretation similar to the AUC and provides an estimation of the probability of correctly ranking two randomly chosen subjects according to their respective risk of experiencing the event of interest.…”

International Validation of the Thrombolysis in Myocardial Infarction (TIMI) Risk Score for Secondary Prevention in Post‐MI Patients: A Collaborative Analysis of the Chronic Kidney Disease Prognosis Consortium and the Risk Validation Scientific Committee

“…For calibration, we plotted predicted risk based on TRS2°P against observed risk in each study and calculated a modified Hosmer‐Lemeshow χ 2 statistic 17. We also calculated the Brier score,18 the average squared deviation between predicted by TRS2°P and observed event rates (a lower score represents better calibration). Observed risk was estimated using the Kaplan–Meier method in each study.…”

The VACS Index Predicts Mortality in a Young, Healthy HIV Population Starting Highly Active Antiretroviral Therapy

“…Cox models were employed, and model performance was evaluated in terms of goodness-of-fit (R-squared), statistical significance (p-values), Akaike information criteria (AIC), and ability to reclassify subjects in given risk groups (net reclassification improvement, NRI). Inverse probability weighting of the cases and controls was used to account for censoring when evaluating the NRI 10,11 .…”