Limitations of the Odds Ratio in Gauging the Performance of a Diagnostic, Prognostic, or Screening Marker

Pepe, Margaret S.; Janes, Holly; Longton, Gary; Leisenring, Wendy M.; Newcomb, Polly A.

doi:10.1093/aje/kwh101

Cited by 1,075 publications

(863 citation statements)

References 25 publications

Supporting

Mentioning

837

Contrasting

Unclassified

Order By: Relevance

“…However, limitations of the use of ROC-AUCs for determination of the predictive value of biomarkers have been addressed [27]. High odds ratios are needed for an individual biomarker to improve the ROC-AUC [28]. Furthermore discrimination, measured by the ROC-AUC, determines the probability of a case being assigned a higher risk than a control, whereas classification assesses the The x-axis represents the predicted probability of death for the model using only the APACHE IV score, the y-axis for the model with APACHE IV and suPAR combined.…”

Section: Discussionmentioning

confidence: 99%

Plasma suPAR as a prognostic biological marker for ICU mortality in ARDS patients

et al. 2015

View full text Add to dashboard Cite

Purpose: We investigated the prognostic value of plasma soluble urokinase plasminogen activator receptor (suPAR) on day 1 in patients with the acute respiratory distress syndrome (ARDS) for intensive care unit (ICU) mortality and compared it with established disease severity scores on day 1. Methods: suPAR was determined batchwise in plasma obtained within 24 h after admission. Results: 632 ARDS patients were included. Significantly (P = 0.02) higher median levels of suPAR were found with increasing severity of ARDS: 5.9 ng/ml [IQR 3.1-12.8] in mild ARDS (n = 82), 8.4 ng/ml .0] in moderate ARDS (n = 333), and 9.0 ng/ml .0] in severe ARDS (n = 217). Non-survivors had higher median levels of suPAR [12.5 ng/ml (IQR 5.1-19.5) vs. 7.4 ng/ml (3.9-13.6), P \ 0.001]. The area under the receiver operator characteristic curve (ROC-AUC) for mortality of suPAR (0.62) was lower than the ROC-AUC of the APACHE IV score (0.72, P = 0.007), higher than that of the ARDS definition classification (0.53, P = 0.005), and did not differ from that of the SOFA score (0.68, P = 0.07) and the oxygenation index (OI) (0.58, P = 0.29). Plasma suPAR did not improve the discrimination of the established disease severity scores, but did improve net reclassification of the APACHE score (29 %), SOFA score (23 %), OI (38 %), and Berlin definition classification (39 %). Conclusion: As a single biological marker, the prognostic value for death of plasma suPAR in ARDS patients is low. Plasma suPAR, however, improves the net reclassification, suggesting a potential role for suPAR in ICU mortality prediction models.

show abstract

Section: Discussionmentioning

confidence: 99%

Plasma suPAR as a prognostic biological marker for ICU mortality in ARDS patients

et al. 2015

View full text Add to dashboard Cite

show abstract

“…However, it has been shown that a risk factor must have a much stronger association with the disease outcome (very large RR) than we ordinarily see in etiologic research for it to be a worthwhile screening test, that is, be capable of efficiently discriminate between persons likely to have the outcome and those who do not (good sensitivity and specificity). (31)(32)(33) Our study has several strengths and limitations. An extensive amount of data were collected for each woman at the baseline and follow-up visits, including the use of osteoporosis medications and HRT throughout the follow-up, which allowed us to examine a large set of potential CRFs for fracture.…”

Section: Discussionmentioning

confidence: 99%

Fracture risk prediction using BMD and clinical risk factors in early postmenopausal women: Sensitivity of the WHO FRAX tool

Trémollières

Pouillès

Drewniak

et al. 2010

Journal of Bone and Mineral Research

179

114

View full text Add to dashboard Cite

The aim of this prospective study was (1) to identify significant and independent clinical risk factors (CRFs) for major osteoporotic (OP) fracture among peri-and early postmenopausal women, (2) to assess, in this population, the discriminatory capacity of FRAX and bone mineral density (BMD) for the identification of women at high risk of fracture, and (3) to assess whether adding risk factors to either FRAX or BMD would improve discriminatory capacity. The study population included 2651 peri-and early postmenopausal women [mean age (AE SD): 54 AE 4 years] with a mean follow-up period of 13.4 years (AE1.4 years). At baseline, a large set of CRFs was recorded, and vertebral BMD was measured (Lunar, DPX) in all women. Femoral neck BMD also was measured in 1399 women in addition to spine BMD. Women with current or past OP treatment for more than 3 months at baseline (n ¼ 454) were excluded from the analyses. Over the follow-up period, 415 women sustained a first low-energy fracture, including 145 major OP fractures (108 wrist, 44 spine, 20 proximal humerus, and 13 hip). In Cox multivariate regression models, only 3 CRFs were significant predictors of a major OP fracture independent of BMD and age: a personal history of fracture, three or more pregnancies, and current postmenopausal hormone therapy. In the subsample of women who had a hip BMD measurement and who were not receiving OP therapy (including hormone-replacement therapy) at baseline, mean FRAX value was 3.8% (AE2.4%). The overall discriminative value for fracture, as measured by the area under the Receiver Operating Characteristic (ROC) curve (AUC), was equal to 0.63 [95% confidence interval (CI) 0.56-0.69] and 0.66 (95% CI 0.60-0.73), respectively, for FRAX and hip BMD. Sensitivity of both tools was low (ie, around 50% for 30% of the women classified as the highest risk). Adding parity to the predictive model including FRAX or using a simple risk score based on the best predictive model in our population did not significantly improve the discriminatory capacity over BMD alone. Only a limited number of clinical risk factors were found associated with the risk of major OP fracture in peri-and early postmenopausal women. In this population, the FRAX tool, like other risk scores combining CRFs to either BMD or FRAX, had a poor sensitivity for fracture prediction and did not significantly improve the discriminatory value of hip BMD alone. ß

show abstract

“…In two studies [21,62], it was necessary to recalculate these figures using the raw numbers presented. Diagnostic odds-ratio's (DOR's) were not calculated because of the recent suggestion that a DOR does not meaningfully describe a marker's ability to classify subjects [41]. A meta-analysis was also not Table 1 Methodological criteria used to assess the quality of studies investigating diagnostic accuracy for subjects with cervical radiculopathy Item 1.…”

Section: Discussionmentioning

confidence: 99%

A systematic review of the diagnostic accuracy of provocative tests of the neck for diagnosing cervical radiculopathy

et al. 2006

View full text Add to dashboard Cite

Clinical provocative tests of the neck, which position the neck and arm inorder to aggravate or relieve arm symptoms, are commonly used in clinical practice in patients with a suspected cervical radiculopathy. Their diagnostic accuracy, however, has never been examined in a systematic review. A comprehensive search was conducted in order to identify all possible studies fulfilling the inclusion criteria. A study was included if: (1) any provocative test of the neck for diagnosing cervical radiculopathy was identified; (2) any reference standard was used; (3) sensitivity and specificity were reported or could be (re-)calculated; and, (4) the publication was a full report. Two reviewers independently selected studies, and assessed methodological quality. Only six studies met the inclusion criteria, which evaluated five provocative tests. In general, Spurling's test demonstrated low to moderate sensitivity and high specificity, as did traction/neck distraction, and Valsalva's maneuver. The upper limb tension test (ULTT) demonstrated high sensitivity and low specificity, while the shoulder abduction test demonstrated low to moderate sensitivity and moderate to high specificity. Common methodological flaws included lack of an optimal reference standard, disease progression bias, spectrum bias, and review bias. Limitations include few primary studies, substantial heterogeneity, and numerous methodological flaws among the studies; therefore, a meta-analysis was not conducted. This review suggests that, when consistent with the history and other physical findings, a positive Spurling's, traction/neck distraction, and Valsalva's might be indicative of a cervical radiculopathy, while a negative ULTT might be used to rule it out. However, the lack of evidence precludes any firm conclusions regarding their diagnostic value, especially when used in primary care. More high quality studies are necessary in order to resolve this issue.

show abstract

Limitations of the Odds Ratio in Gauging the Performance of a Diagnostic, Prognostic, or Screening Marker

Cited by 1,075 publications

References 25 publications

Plasma suPAR as a prognostic biological marker for ICU mortality in ARDS patients

Plasma suPAR as a prognostic biological marker for ICU mortality in ARDS patients

Fracture risk prediction using BMD and clinical risk factors in early postmenopausal women: Sensitivity of the WHO FRAX tool

A systematic review of the diagnostic accuracy of provocative tests of the neck for diagnosing cervical radiculopathy

Contact Info

Product

Resources

About