Wilcoxon‐based group sequential designs for comparison of areas under two correlated ROC curves

Abstract

Clinical studies to evaluate the relative accuracies of two diagnostic modalities via their receiver operating characteristic (ROC) curves are currently conducted using fixed sample designs: cases are accrued until a predetermined sample size is achieved and, at that point, the areas under the ROC curves are computed and compared (Radiology 1982; 143:29-36; Radiology 1983; 148:839-843). In prospective ROC studies (Radiology 1990; 175:571-575), participants are recruited from a clinically defined cohort and diagnostic test information is obtained and interpreted in advance of ascertaining the definitive proof of diagnosis ('gold standard'). In retrospective studies, cases are selected from a set of patient records and their diagnostic tests are interpreted without knowledge of the 'gold standard'. The conduct of ROC studies requires considerable effort and resources, particularly for the collection of 'gold standard' information. Thus, it is highly desirable to search for designs which are more efficient than using a fixed sample. In this paper, we discuss the formulation and application of group sequential designs (GSDs) to comparative ROC studies based on non-parametric Wilcoxon estimators of the area under the ROC curves. The approach is applicable to comparisons of ROC curve areas of two tests measured on either continuous or ordinal scales on same cases ('paired' designs) with one reader. The adoption of GSDs may lead to substantial savings in the number of required cases, thus resulting in both time and resource use efficiency.

Design of Scoring Models for Trustworthy Risk Prediction in Critical Patients

“… Training is stopped when the cumulative increment in A U C o b t a i n e d i n f i v e consecutive steps is less than 1%. This rather soft stopping criterion is used instead of well-established statistical methods (Zhou et al, 2008) to avoid selecting too few predictors, which reduces the possibility of associating an effective probability of AHE with each integer score.  A testing dataset of the same size as the training set is used to evaluate model generalisation and to guide conclusive selection of the optimal predictor set.…”

Sequential diagnostic trial designs

“…We also calculate the variance of \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$\hat{\Delta}$ \end{document} from the explicit form derived by Wieand et al2 and Tang et al3 It is showed that for the large sample size, the Δ‐statistic \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$\hat\Delta=\hat\theta_1-\hat\theta_2$\end{document} is equivalent to the summation of i.i.d. random variables 5. If we define the B ‐value as \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$B(\tau_{i})=\hat{\Delta}\sqrt{\tau_{i}}/\sigma_{i}$ \end{document}, where τ i is the ratio of the Fisher information at the i th interim ( I i ) to the Fisher information at the K th look ( I K ) for \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$I_{i}=1/\sigma^{2}_{i}$ \end{document} and \documentclass{article}\usepackage{amsmath}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{amsfonts}\pagestyle{empty}\begin{document}$I_{K}=1/\sigma^{2}_{K}$ \end{document}, respectively.…”

“…In addition, a nonparametric GSD model is proposed to compare the diagnostic accuracy of two markers 5. In their article, the following Delong's statistic is used to estimate the difference of two AUCs6: where The above two GSD methods are for comparing AUCs.…”

Group sequential comparison of positive predictive value curves for correlated biomarker data

“…In diagnostic medicine, sequential testing procedures mainly focus on the discriminating accuracy of diagnostic tests. GSDs have been proposed to compare the receiver operating characteristics (ROC) curves and their areas using binormal distributions and empirical distributions ROC curves . More recently, researchers studied the GSD for predictive accuracy.…”