Validation of an internationally derived patient severity phenotype to support COVID-19 analytics from electronic health record data

Abstract

Introduction The Consortium for Clinical Characterization of COVID-19 by EHR (4CE) is an international collaboration addressing COVID-19 with federated analyses of electronic health record (EHR) data. Objective We sought to develop and validate a computable phenotype for COVID-19 severity. Methods Twelve 4CE sites participated. First we developed an EHR-based severity phenotype consisting of six code classes, and we validated it on patient hospitalization data from the 12 4CE clinical sites against the outcomes of ICU admission and/or death. We also piloted an alternative machine-learning approach and compared selected predictors of severity to the 4CE phenotype at one site. Results The full 4CE severity phenotype had pooled sensitivity of 0.73 and specificity 0.83 for the combined outcome of ICU admission and/or death. The sensitivity of individual code categories for acuity had high variability - up to 0.65 across sites. At one pilot site, the expert-derived phenotype had mean AUC 0.903 (95% CI: 0.886, 0.921), compared to AUC 0.956 (95% CI: 0.952, 0.959) for the machine-learning approach. Billing codes were poor proxies of ICU admission, with as low as 49% precision and recall compared to chart review. Discussion We developed a severity phenotype using 6 code classes that proved resilient to coding variability across international institutions. In contrast, machine-learning approaches may overfit hospital-specific orders. Manual chart review revealed discrepancies even in the gold-standard outcomes, possibly due to heterogeneous pandemic conditions. Conclusion We developed an EHR-based severity phenotype for COVID-19 in hospitalized patients and validated it at 12 international sites.

Multinational characterization of neurological phenotypes in patients hospitalized with COVID-19

“…Our other major finding indicates that a significantly higher proportion of hospitalized patients with severe COVID-19 (based on a computational phenotyping algorithm of COVID-19 severity that our group previously published 30 ) had disorders of consciousness and other disorders of the brain, encephalitis and/or myelitis, cerebrovascular events and myopathy when compared to patients who never had severe disease. Beyond corroborating prior reports 19 , 23 – 25 , our findings highlighted similar patterns across geographically diverse multinational healthcare systems using a standardized approach.…”

“…2 A). The study captured a broad range of proportions of severe COVID-19 cases (based on the internationally validated 4CE COVID-19 severity criteria 30 ) across this multinational network of healthcare systems. There was no clear relationship between COVID-19 severity and median age (Fig.…”

“…To assess the association with COVID-19 severity, we next used Fisher’s exact test to examine the enrichment or depletion of each neurological condition among patients hospitalized for COVID-19 who ever experienced severe disease based on the published 4CE COVID-19 severity criteria 30 , using those who never experienced severe disease as the comparator (Fig. 4 ).…”

“…Leveraging the highly scalable, federated, multinational networks of the 4CE consortium, we computed the prevalence of a wide range of central and peripheral neurological conditions in hospitalized patients with reverse transcription polymerase chain reaction (PCR)-confirmed SARS-CoV-2 infection by healthcare system and by country. We also compared the differences in the prevalence of neurological conditions between patients with and those without severe COVID-19 based on the internationally validated 4CE severity criteria 30 .…”

Pragmatic MDR: a metadata repository with bottom-up standardization of medical metadata through reuse

“…Standardizing and reusing such metadata definitions has two major advantages. First, it yields harmonized data sets that allow data exchange between institutions [3,4] and facilitate data analyses, such as multi-site phenotyping [5] or machine learning [6]. Second, medical documentation does not have to be developed from scratch reducing costs [7].…”