Comparative Effectiveness of Knowledge Graphs- and EHR Data-Based Medical Concept Embedding for Phenotyping

Abstract

Objective: Concept identification is a major bottleneck in phenotyping. Properly learned medical concept embeddings (MCEs) have semantic meaning of the medical concepts, thus useful for feature engineering in phenotyping tasks. The objective of this study is to compare the effectiveness of MCEs learned by using knowledge graphs and EHR data for facilitating high-throughput phenotyping. Materials and Methods: We investigated four MCEs learned from different data sources and methods. Knowledge-graphs were obtained from the Observational Medical Outcomes Partnership (OMOP) common data model. Medical concept co-occurrence statistics were obtained from Columbia University Irving Medical Center's (CUIMC) OMOP database. Two embedding methods, node2vec and GloVe, were used to learn embeddings for medical concepts. We used phenotypes with their corresponding concepts generated and validated by the Electronic Medical Records and Genomics (eMERGE) network to evaluate the performance of learned MCEs in identifying phenotype-relevant concepts. Results: Precision@k% and Recall@k% in identifying phenotype-relevant concepts based on a single concept and multiple seed concepts were used to evaluate MCEs. Recall@500% and Precision@500% based on a single seed concept of MCE learned using the enriched knowledge graph were 0.64 and 0.13, compared to Recall@500% and Precision@500% of MCE learned using the hierarchical knowledge graph (0.61 and 0.12), 5-year windowed EHR (0.51 and 0.10), and visit-windowed EHR (0.46 and 0.09). Conclusion: Medical concept embedding enables scalable identification of phenotype-relevant medical concepts, thereby facilitating high-throughput phenotyping. Knowledge graphs constructed by hierarchical relationships among medical concepts learn more effective MCEs, highlighting the need of more sophisticated use of big data to leverage MCEs for phenotyping.

Phe2vec: Automated disease phenotyping based on unsupervised embeddings from electronic health records

“…Since the number of concepts in a PheKB definition (i.e., recall level R) varies across diseases, we evaluated different neighbors with sizes equal to percentages of R of the corresponding disease (R%). 18 We measured precision and recall per disease, where precision is the number of correct positive results divided by the number of all positive results, and recall is the number of correct positive results divided by the number of positive results that should have been returned. Results averaged over the ten diseases are reported in Figure 3 .…”

“… 8 Previous work in this domain used supervised and unsupervised machine learning to derive phenotypes for several diseases, with different strengths and limitations (see literature review in Note S1). 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 Supervised models rely on classifiers based on manually labeled gold standards for each specific disease, which is time-consuming and not scalable. Unsupervised approaches discover phenotypes purely from the data, trying to aggregate medical concepts commonly appearing together in the patient records.…”

Severity Prediction for COVID-19 Patients via Recurrent Neural Networks

“…The pre-trained embedding captures the relationships between the medical concepts since GloVe utilizes the global co-occurrence matrix of concepts for its training, where the co-occurrence matrix is calculated based on the concept co-occurrence in every patients’ visit. In the literature, the dimensionality of the embedding is generally set between 100–500 for medical concept vocabularies with sizes from a few hundred to tens of thousands of concepts 11 , 20 , therefore we set the dimensionality of the pre-trained embedding and randomly initialized embedding to 128. The embedding layer was fine-tuned jointly with the prediction task of the model.…”

Phe2vec: Automated Disease Phenotyping based on Unsupervised Embeddings from Electronic Health Records

“…To assess the performance of Phe2vec in building phenotypes, for each disease, we evaluated the overlap between the medical concepts retrieved from the seed neighbors and those in PheKB. Since the number of concepts in a PheKB definition (i.e., recall level R) varies across diseases, we evaluated different neighbors with sizes equal to percentages of R of the corresponding disease (R%) [18]. We measured precision and recall per disease, where precision is the number of correct positive results divided by the number of all positive results, and recall is the number of correct positive results divided by the number of positive results that should have been returned.…”

“…Automated phenotyping provides a more rapid and scalable alternative if it can achieve the same robustness as rule-based algorithms [8]. Previous work in this domain used supervised and unsupervised machine learning to derive phenotypes for several diseases, with different strengths and limitations [9][10][11][12][13][14][15][16][17][18][19][20][21][22]. Supervised models rely on classifiers based on manually-labeled gold standards for each specific disease, which is time-consuming and not scalable.…”