Comparative analysis, applications, and interpretation of electronic health record-based stroke phenotyping methods

Abstract

Background Accurate identification of acute ischemic stroke (AIS) patient cohorts is essential for a wide range of clinical investigations. Automated phenotyping methods that leverage electronic health records (EHRs) represent a fundamentally new approach cohort identification without current laborious and ungeneralizable generation of phenotyping algorithms. We systematically compared and evaluated the ability of machine learning algorithms and case-control combinations to phenotype acute ischemic stroke patients using data from an EHR. Materials and methods Using structured patient data from the EHR at a tertiary-care hospital system, we built and evaluated machine learning models to identify patients with AIS based on 75 different case-control and classifier combinations. We then estimated the prevalence of AIS patients across the EHR. Finally, we externally validated the ability of the models to detect AIS patients without AIS diagnosis codes using the UK Biobank. Results Across all models, we found that the mean AUROC for detecting AIS was 0.963 ± 0.0520 and average precision score 0.790 ± 0.196 with minimal feature processing. Classifiers trained with cases with AIS diagnosis codes and controls with no cerebrovascular disease codes had the best average F1 score (0.832 ± 0.0383). In the external validation, we found that the top probabilities from a model-predicted AIS cohort were significantly enriched for AIS patients without AIS diagnosis codes (60–150 fold over expected). Conclusions Our findings support machine learning algorithms as a generalizable way to accurately identify AIS patients without using process-intensive manual feature curation. When a set of AIS patients is unavailable, diagnosis codes may be used to train classifier models.

Benchmarking Polygenic Risk Score Model Assumptions: towards more accurate risk assessment

“…For example, 148 individuals with diabetes report HbA1c levels less than 35 mmol/mol, even though 48 mmol/mol is traditionally the diagnosis cut-off [59]. Several recent investigations have proven the practicality of using a wide range of diverse features to form accurate disease predictions that are robust to any single error in the electronic health records [28, 29]. However, these predictions are still far from perfect.…”

“…If a prediction derived from the patient data confidently disagrees with the disease label derived from electronic health records, then the corresponding suspect individual could be removed from the analysis. Previous studies have demonstrated with evidence the accuracy of disease predictions derived from large datasets of patient data [28, 29], but to the best of our knowledge they have not assessed whether using disease predictions instead of direct disease labels effects the predictions of polygenic risk score models.…”

An interpretable outcome prediction model based on electronic health records and hierarchical attention

“…With the broad adoption of EHR systems and the collection of large amount of EHR data, EHR data mining has become a hot research topic. Typical tasks of EHR data mining include medical concept learning, 32 patient subtyping, 33 phenotyping, 34 disease progression analysis, 35 and outcome prediction. Outcome prediction, which aims to predict the future health state of a patient, is one of the most important tasks in EHR data mining.…”

Application of concise machine learning to construct accurate and interpretable EHR computable phenotypes

“…[7] Due to the high-dimensional, messy, noisy data that constitute EHRs, many studies have developed ensemble or deep learning methods to train accurate models. [10–17] Algorithms employed in these studies (e.g. random forests and neural networks) generally can perform well in classification but are often limited in their interpretability , a subjective concept defined as the extent to which a model can be understood and/or its behavior interpreted by a user.…”