Methods of evaluating and comparing the performance of diagnostic tests are of increasing importance as new tests are developed and marketed. When a test is based on an observed variable that lies on a continuous or graded scale, an assessment of the overall value of the test can be made through the use of a receiver operating characteristic (ROC) curve. The curve is constructed by varying the cutpoint used to determine which values of the observed variable will be considered abnormal and then plotting the resulting sensitivities against the corresponding false positive rates. When two or more empirical curves are constructed based on tests performed on the same individuals, statistical analysis on differences between curves must take into account the correlated nature of the data. This paper presents a nonparametric approach to the analysis of areas under correlated ROC curves, by using the theory on generalized U-statistics to generate an estimated covariance matrix.

Risk-adjustment and provider profiling have become common terms as the medical profession attempts to measure quality and assess value in health care. One of the areas of care most thoroughly developed in this regard is quality assessment for coronary artery bypass grafting (CABG). Because in-hospital mortality following CABG has been studied extensively, risk-adjustment mechanisms are already being used in this area for provider profiling. This study compares eight different risk-adjustment methods as applied to a CABG surgery population of 28 providers. Five of the methods use an external risk-adjustment algorithm developed in an independent population, while the other three rely on an internally developed logistic model. The purposes of this study are to: (i) create a common metric by which to display the results of these various risk-adjustment methodologies with regard to dichotomous outcomes such as in-hospital mortality, and (ii) to compare how these risk-adjustment methods quantify the 'outlier' standing of providers. Section 2 describes the data, the external and internal risk-adjustment algorithms, and eight approaches to provider profiling. Section 3 then demonstrates the results of applying these methods on a data set specifically collected for quality improvement.