Time-dependent ROC curve analysis in medical research: current methods and applications

Abstract

BackgroundROC (receiver operating characteristic) curve analysis is well established for assessing how well a marker is capable of discriminating between individuals who experience disease onset and individuals who do not. The classical (standard) approach of ROC curve analysis considers event (disease) status and marker value for an individual as fixed over time, however in practice, both the disease status and marker value change over time. Individuals who are disease-free earlier may develop the disease later due to longer study follow-up, and also their marker value may change from baseline during follow-up. Thus, an ROC curve as a function of time is more appropriate. However, many researchers still use the standard ROC curve approach to determine the marker capability ignoring the time dependency of the disease status or the marker.MethodsWe comprehensively review currently proposed methodologies of time-dependent ROC curves which use single or longitudinal marker measurements, aiming to provide clarity in each methodology, identify software tools to carry out such analysis in practice and illustrate several applications of the methodology. We have also extended some methods to incorporate a longitudinal marker and illustrated the methodologies using a sequential dataset from the Mayo Clinic trial in primary biliary cirrhosis (PBC) of the liver.ResultsFrom our methodological review, we have identified 18 estimation methods of time-dependent ROC curve analyses for censored event times and three other methods can only deal with non-censored event times. Despite the considerable numbers of estimation methods, applications of the methodology in clinical studies are still lacking.ConclusionsThe value of time-dependent ROC curve methods has been re-established. We have illustrated the methods in practice using currently available software and made some recommendations for future research.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-017-0332-6) contains supplementary material, which is available to authorized users.

Very high pacing thresholds during long‐term follow‐up predicted by a combination of implant pacing threshold and impedance in leadless transcatheter pacemakers

“…The performance of electrical parameters at implant (area under the curve [AUC], SEN, specificity [SPE], and positive and negative predictive values [PPV and NPV, respectively]) was tested to predict long‐term development of VHPT. Time‐dependent receiver operating characteristic (ROC) curve analyses were used to account for variable follow‐up and censoring 7 . Specifically, the cumulative SPE‐dynamic SPE definition was used to estimate performance at 1, 12, 24, 36, and 48 months postimplant ( timeROC package in R v3.6.0).…”

Prognostic and predictive implications of sarcopenia in Western patients undergoing gastric resections for carcinoma of the stomach

“…The differences in proportions between groups were evaluated using the χ 2 test, and the Mann‐Whitney U test was used to detect differences in quantitative variables. Parameters obtained from cross‐section imaging (relative tissue areas of adipose tissue compartments, the density of adipose and skeletal tissues, and SMI) were evaluated using receiver operating characteristic (ROC) curves and time‐dependent ROC curve analysis to identify the optimum predictors of major postoperative complications (Clavien‐Dindo classification ≥3a) and survival, respectively . Potential risk factors for these complications were evaluated by univariate analysis using cross‐tabulations and a backward stepwise logistic regression model.…”

Comparison of pre‐ and post‐bronchodilator lung function as predictors of mortality: The HUNT Study

“…21,22 We calculated time-dependent AUC to compare the discrimination ability of pre-BD and post-BD lung function to predict mortality. [23][24][25][26] The incident/dynamic (I/D) AUC models account for incident cases at time t and dynamic controls, which means it characterizes the time-varying performance without selecting a particular timeframe over which cases accrue, whereas cumulative/dynamic (C/D) AUC models account for cumulative cases at time t and dynamic controls. 25,27 We compared the AUC for crude models because the clinical decision usually does not explicitly take other factors into account.…”