The effects of misclassification in routine healthcare databases on the accuracy of prognostic prediction models: a case study of the CHA2DS2-VASc score in atrial fibrillation

Doorn, Sander van; Brakenhoff, TB; Moons, Kgm; Rutten, Frans H.; Hoes, Arno W.; Groenwold, Rhh; Geersing, Geert Jan

doi:10.1186/s41512-017-0018-x

Cited by 13 publications

(15 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, it is acknowledged that this is not always possible, particularly for conditions that require a consensus diagnosis based on all available patient information [143]. It is well known that misclassification in the outcome variable may cause serious problems with prediction accuracy [144, 145].…”

Section: Stages In the Development Of Clinical Prediction Rulesmentioning

confidence: 99%

Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature

et al. 2019

View full text Add to dashboard Cite

Clinical prediction rules (CPRs) that predict the absolute risk of a clinical condition or future outcome for individual patients are abundant in the medical literature; however, systematic reviews have demonstrated shortcomings in the methodological quality and reporting of prediction studies. To maximise the potential and clinical usefulness of CPRs, they must be rigorously developed and validated, and their impact on clinical practice and patient outcomes must be evaluated. This review aims to present a comprehensive overview of the stages involved in the development, validation and evaluation of CPRs, and to describe in detail the methodological standards required at each stage, illustrated with examples where appropriate. Important features of the study design, statistical analysis, modelling strategy, data collection, performance assessment, CPR presentation and reporting are discussed, in addition to other, often overlooked aspects such as the acceptability, cost-effectiveness and longer-term implementation of CPRs, and their comparison with clinical judgement. Although the development and evaluation of a robust, clinically useful CPR is anything but straightforward, adherence to the plethora of methodological standards, recommendations and frameworks at each stage will assist in the development of a rigorous CPR that has the potential to contribute usefully to clinical practice and decision-making and have a positive impact on patient care.

show abstract

Section: Stages In the Development Of Clinical Prediction Rulesmentioning

confidence: 99%

Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Sixty studies used medical record review as a reference standard. 22,23,[25][26][27][28][29][30][31][32][33][34][35][36][37][38][39]42,43,45,46,49,50,54,55,[57][58][59][60][61][62][63]69,72, Twenty studies validated HF, 24,[28][29][30]33,43,46,54,59,65,67,77,82,83,85,88,[94][95][96]…”

Section: Study Characteristicsmentioning

confidence: 99%

“…Fourteen studies validated secondary care EHRs 24,[28][29][30]33,43,54,59,65,67,77,83,85,88 and six studies validated primary care EHRs. 46,82,[94][95][96][97] Medical record review was used as the reference standard in all but three studies. 24,65,97 Heart Failure Validation Results…”

Section: Heart Failure Study Characteristicsmentioning

confidence: 99%

<p>Validity of Acute Cardiovascular Outcome Diagnoses Recorded in European Electronic Health Records: A Systematic Review</p>

Davidson¹,

Banerjee²,

Muzambi³

et al. 2020

CLEP

View full text Add to dashboard Cite

Background: Electronic health records are widely used in cardiovascular disease research. We appraised the validity of stroke, acute coronary syndrome and heart failure diagnoses in studies conducted using European electronic health records. Methods: Using a prespecified strategy, we systematically searched seven databases from dates of inception to April 2019. Two reviewers independently completed study selection, followed by partial parallel data extraction and risk of bias assessment. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value estimates were narratively synthesized and heterogeneity between sensitivity and PPV estimates were assessed using I 2. Results: We identified 81 studies, of which 20 validated heart failure diagnoses, 31 validated acute coronary syndrome diagnoses with 29 specifically recording estimates for myocardial infarction, and 41 validated stroke diagnoses. Few studies reported specificity or negative predictive value estimates. Sensitivity was ≤66% in all but one heart failure study, ≥80% for 91% of myocardial infarction studies, and ≥70% for 73% of stroke studies. PPV was ≥80% in 74% of heart failure, 88% of myocardial infarction, and 70% of stroke studies. PPV by stroke subtype was variable, at ≥80% for 80% of ischaemic stroke but only 44% of haemorrhagic stroke. There was considerable heterogeneity (I 2 >75%) between sensitivity and PPV estimates for all diagnoses. Conclusion: Overall, European electronic health record stroke, acute coronary syndrome and heart failure diagnoses are accurate for use in research, although validity estimates for heart failure and individual stroke subtypes were lower. Where possible, researchers should validate data before use or carefully interpret the results of previous validation studies for their own study purposes.

show abstract

“…Because time from injury to ambulance arrival and hospital admission was missing for nearly half of the patients, we imputed these data. Misclassi cation of time to treatment could affect our estimate of the net bene t. [22] The estimates of deaths avoided are unlikely to be generalizable since they depend on the risk of death, which may vary in different settings.…”

Section: Strengths and Limitationsmentioning

confidence: 99%

Validation of the BATT score for prehospital risk stratification of traumatic haemorrhagic death: Usefulness for tranexamic acid treatment criteria.

Ageron

Coats

Darioli

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: Tranexamic acid reduces surgical blood loss and reduces deaths from bleeding in trauma patients. Tranexamic acid must be given urgently, preferably by paramedics at the scene of the injury or in the ambulance. We developed a simple score (Bleeding Audit Triage Trauma score) to predict death from bleeding.Methods: We conducted an external validation of the BATT score using data from the UK Trauma Audit Research Network (TARN) from 1st January 2017 to 31st December 2018. We evaluated the impact of tranexamic acid treatment thresholds in trauma patients.Results: We included 104,862 trauma patients with an injury severity score of 9 or above. Tranexamic acid was administered to 9,915 (9%) patients. Of these 5,185 (52%) received prehospital tranexamic acid. The BATT score had good accuracy (Brier score=6%) and good discrimination (C-statistic 0.90; 95% CI 0.89-0.91). Calibration in the large showed no substantial difference between predicted and observed death due to bleeding (1.15% versus 1.16%; P=0.81). Pre-hospital tranexamic acid treatment of trauma patients with a BATT score of 2 or more would avoid 210 bleeding deaths by treating 61,598 patients instead of avoiding 55 deaths by treating 9,915 as currently. Conclusion: The BATT score identifies trauma patient at risk of significant haemorrhage. A score of 2 or more would be an appropriate threshold for pre-hospital tranexamic acid treatment.

show abstract

The effects of misclassification in routine healthcare databases on the accuracy of prognostic prediction models: a case study of the CHA2DS2-VASc score in atrial fibrillation

Cited by 13 publications

References 28 publications

Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature

Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature

<p>Validity of Acute Cardiovascular Outcome Diagnoses Recorded in European Electronic Health Records: A Systematic Review</p>

Validation of the BATT score for prehospital risk stratification of traumatic haemorrhagic death: Usefulness for tranexamic acid treatment criteria.

Contact Info

Product

Resources

About