Derivation and Validation of a Risk Factor Model to Identify Medical Inpatients at Risk for Venous Thromboembolism

Abstract

Background: Venous thromboembolism (VTE) prophylaxis is recommended for hospitalized medical patients at high risk for VTE. Multiple risk assessment models exist, but few have been compared in large data sets. Methods: We constructed a derivation cohort using 6 years of data from 13 hospitals to identify risk factors associated with developing VTE within 14 days of admission. VTE was identified using a complex algorithm combining administrative codes and clinical data. We developed a multivariable prediction model and applied it to 2 validation cohorts: a temporal cohort, including two additional years and a cross-validation, in which we refit the model excluding one hospital at a time, and applied the refitted model to the holdout hospital. Performance was evaluated using the C-statistic. Results: The derivation cohort included 160,928 patients with a 14-day VTE rate of 0.79%. The final multivariable model contained 13 patient risk factors. The model had an optimism corrected C-statistic of 0.80 and good calibration. The temporal validation cohort included 55,301 patients, with a VTE rate of 0.74%. Based on the c-statistic, the Cleveland Clinic Model (CCM) outperformed the Padua model (0.76 vs. 0.72, p<0.01). The CCM was more sensitive (65.8% vs. 60.4%, p=0.05) and more specific (74.9% vs. 71.4%, p<.001), with higher positive (1.9% vs. 1.5%, p<.001) and negative predictive values (99.7% vs. 99.6%, p=0.01). C-statistics for the CCM at individual hospitals ranged from 0.64 to 0.76. Conclusion: A new VTE risk assessment model outperformed the Padua model. After further validation it could be recommended for widespread use.

Trends and Risk Factors for Venous Thromboembolism Among Hospitalized Medical Patients

“… 7 , 15 , 16 , 17 , 18 , 19 , 20 Our logic requiring either a definitive PE diagnosis on CT or a combination of diagnosis and treatment indications, as well as the exclusion of admissions in which there was an HA-VTE event or administration of therapeutic anticoagulation in the first 48 hours, appears to have been associated with more accurate detection of true HA-VTE than reliance on diagnosis codes alone. 16 , 22 The inclusion of HA-VTE up to 90 days after discharge may have been especially important given findings by us and others 15 , 16 that most HA-VTE events happened after discharge. Additionally, unlike other studies, 20 , 22 , 47 to better simulate a common clinical scenario of clinician risk assessment at time of admission and reduce various forms of biases, our algorithm was permitted to capture only data associated with risk factors as they were recorded at the exact time of admission.…”

“…To optimize the accuracy of our HA-VTE definition, we used multiple criteria to establish an HA-VTE event, adapting methods described in prior studies. 16 , 22 We identified an HA-VTE event if there was a definite finding of PE on a computed tomography (CT) scan based on inclusion of specific tags in the CT report (eAppendix in the Supplement ) or if an admission was associated with 1 or more indications of VTE diagnosis (new International Classification of Diseases, Ninth Revision [ ICD-9 ] or International Statistical Classification of Diseases and Related Health Problems, Tenth Revision [ ICD-10 ] codes for VTE, an “abnormal” vascular ultrasound, or a CT scan suspicious for PE) and 1 or more indications of VTE treatment, which could include the following: (1) first encounter with or new referral to the anticoagulation clinic for VTE, (2) new filled therapeutic anticoagulation prescription, (3) new ICD-9 or ICD-10 diagnosis code for long-term anticoagulation, (4) placement of a new inferior vena cava filter, or (5) in-hospital death during the index admission after starting therapeutic anticoagulation (see eFigure 2 in the Supplement for a graphical representation of HA-VTE criteria logic, eTables 1 and 2 in the Supplement for prevalence of specific HA-VTE criteria, and the eAppendix in the Supplement for detailed specifications). We further categorized HA-VTE events as DVT (ie, abnormal vascular ultrasound, new ICD-9 or ICD-10 code for DVT, or both), PE (ie, CT scan definitive or suspicious for PE, new ICD-9 or ICD-10 code for PE, or both), DVT and PE, or unknown VTE type if none of these subclassifications applied (eAppendix in the Supplement ).…”

“…with iterative corrections to HA-VTE criteria until each review exhibited an accuracy of 90% or higher. 22 Once criteria were finalized, we manually reviewed a random sample of 200 EHRs of HA-VTE events based on the algorithm and found that the approach yielded a 95% true positive rate (190 of 200 events [95.0%]). Because no standardized registry for confirmed HA-VTE events existed and because HA-VTE events were uncommon, we also explored false-negative rates and missed cases by reviewing a random sample of 100 admissions in which 2 or more HA-VTE criteria were met but were insufficient to meet HA-VTE logic definitions (eg, multiple indications of VTE treatment without an indication of diagnosis).…”

“…To optimize the accuracy of our HA-VTE definition, we used multiple criteria to establish an HA-VTE event, adapting methods described in prior studies. 16,22 We identified an HA-VTE event if there was a definite finding of PE on a computed tomography (CT) scan based on inclusion of specific tags in the CT report (eAppendix in the Supplement) or if an admission was associated with 1 or more…”

“…3 We conducted retrospective manual EHR review for each risk factor with iterative corrections made until we achieved an accuracy of 90% or higher. 22 In addition, for Padua Predictive Score elements, 24 we prospectively validated each element in a sample of 75 consecutive admissions in October and November 2021 based on manual EHR review by internal medicine residents (including J.X. ); this exhibited 90% or greater accuracy (ie, no more than 7of 75 incongruent admissions) for each individual criterion and for the overall score.…”