Evaluation of a Prediction Model for the Development of Atrial Fibrillation in a Repository of Electronic Medical Records

Abstract

Importance Atrial fibrillation contributes to substantial morbidity, mortality, and healthcare expenditures. Accurate prediction of incident atrial fibrillation would enhance patient management and potentially improve outcomes. Objective We aimed to validate the atrial fibrillation risk prediction model originally developed by the CHARGE-AF investigators utilizing a large repository of electronic medical records. Design Using a database of de-identified medical records, we conducted a retrospective electronic medical record study of subjects without atrial fibrillation followed in Internal Medicine outpatient clinics at our institution. Individuals were followed for incident atrial fibrillation from 2005 until 2010. Adjusting for differences in baseline hazard, we applied the CHARGE-AF Cox proportional hazards model regression coefficients to our cohort. A simple version of the model, with no ECG variables was also evaluated. Setting Outpatient clinics at a large academic medical center. Participants 33,494 subjects of age ≥40 years, white or African American, and no previous history of atrial fibrillation. Predictors Predictors in the model included age, race, height, weight, systolic and diastolic blood pressure, treatment for hypertension, smoking status, diabetes, heart failure, history of myocardial infarction, left ventricular hypertrophy, and PR interval. Main outcome Incident atrial fibrillation. Results The median age was 57 years (25th to 75th percentile: 49 to 67), 57% of patients were women, 85.7% were white, 14.3% were African American. During the mean follow-up period of 4.8 ± 0.85 years, 2455 (7.3%) subjects developed atrial fibrillation. Both models had poor calibration in our cohort, with under-prediction of AF among low-risk subjects and over-prediction of AF among high-risk subjects. The full CHARGE-AF model had a C-index of 0.71 (95% confidence interval [CI]: 0.70 to 0.72) in our cohort. The simple model had similar discrimination (C-index: 0.71, 95% CI: 0.70 to 0.72, P = 0.71 for difference between models). Conclusions and Relevance Despite reasonable discrimination, the CHARGE-AF models showed poor calibration in our EMR cohort. Our study highlights the difficulties of applying a risk model derived from prospective cohort studies to an EMR cohort and suggests that these AF risk prediction models be used with caution in the EMR setting. Future risk models may need to be developed and validated within EMR cohorts.

A disease-specific comorbidity index for predicting mortality in patients admitted to hospital with a cardiac condition

“…Second, these risk models could be used as a research tool to adjust for case mix, thus providing a standardized comparison between different study populations, which is relevant for both researchers and health care administrators. 32 However, such speculations are hypothesis generating only, and the specific merits of the application of the cardiac-specific comorbidity index should be evaluated by further research.…”

Risk Score for Prediction of 10-Year Atrial Fibrillation: A Community-Based Study

“…These AF risk scores derived from prospective cohort studies may have poor calibration. 12 The widespread availability of electronic health records (EHRs) that include real-world data from large numbers of patients offers an opportunity for the development and validation of risk models using large numbers of individuals, frequently greatly exceeding sample sizes available in individual trials or prospective cohorts of limited generalizability. 13 Implicit in the use of an EHR-based score, is the availability of the data to the primary physician to support decision making without further testing.…”

Is Achieving the American Heart Association's Life Simple 7 Goals Sufficient to Reduce the Burden of Atrial Fibrillation? No Simple Answers

“…With these challenges in mind, the electronic medical record repositories may be of value in assessing response to risk factor modification. These can serve as an inexpensive and efficient complement to community cohort studies not only for the development of prediction models17 but also prospectively evaluating AF risk factor modification. Furthermore, given that electronic medical records are integrated into clinical practice, prediction models could be incorporated into these systems to prospectively identify individuals at high risk for AF with the ultimate goal of developing individualized preventive strategies.…”