Clinical Concept Embeddings Learned from Massive Sources of Multimodal Medical Data

Beam, Andrew L.; Kompa, Benjamin; Schmaltz, Allen; Fried, Inbar; Weber, Griffin M; Palmer, Nathan; Xu, Shengyuan; Cai, Tianxi; Kohane, Isaac S.

doi:10.48550/arxiv.1804.01486

Cited by 10 publications

(23 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…General patterns are captured during pre-trained processes and can be "transferred" into new prediction tasks. There also exist some pre-trained embeddings of biomedical entities (Choi et al, 2016;Beam et al, 2018) which allow us to adopt similar ideas of "transfer learning" to learn graph embeddings. We can initialize the embedding vector for each node on a graph with its pre-trained embedding (e.g., by looking for the corresponding entity in (Choi et al, 2016;Beam et al, 2018)) rather than by random initialization, and then continue training various graph embedding methods as before (which is often referred to as "fine-tuning").…”

Section: Methodsmentioning

confidence: 99%

“…There also exist some pre-trained embeddings of biomedical entities (Choi et al, 2016;Beam et al, 2018) which allow us to adopt similar ideas of "transfer learning" to learn graph embeddings. We can initialize the embedding vector for each node on a graph with its pre-trained embedding (e.g., by looking for the corresponding entity in (Choi et al, 2016;Beam et al, 2018)) rather than by random initialization, and then continue training various graph embedding methods as before (which is often referred to as "fine-tuning"). The pre-trained embeddings can be seen as "coarse embeddings" since they are usually pre-trained on a large general corpus and have not been optimized for downstream tasks yet.…”

Section: Methodsmentioning

confidence: 99%

“…We conduct experiment with this transfer learning idea on the "CTD DDA" graph. As seen from Table S3 in the Supplementary Materials, the link prediction performance has been improved using the pre-trained embeddings from (Beam et al, 2018).…”

Section: Methodsmentioning

confidence: 99%

“…In the past few years, the increase of clinical texts have been encouraging data-driven models for improving the patient personal care and help clinical decision (Mullenbach et al, 2018). However, due to the privacy and security concerns, the access to raw clinical texts is often limited (Finlayson et al, 2014;Ta et al, 2018;Beam et al, 2018). To facilitate research on clinical texts, a popular substitute strategy for releasing raw clinical texts is to extract medical terms and their aggregated co-occurrence counts from the clinical texts (Finlayson et al, 2014;Ta et al, 2018).…”

Section: Node Classificationmentioning

confidence: 99%

See 3 more Smart Citations

Graph Embedding on Biomedical Networks: Methods, Applications, and Evaluations

Yue,

Wang,

Huang

et al. 2019

Preprint

View full text Add to dashboard Cite

Motivation:Graph embedding learning which aims to automatically learn low-dimensional node representations has drawn increasing attention in recent years. To date, most recent graph embedding methods are mainly evaluated on social and information networks and have not yet been comprehensively studied on biomedical networks under systematic experiments and analyses. On the other hand, for a variety of biomedical network analysis tasks, traditional techniques such as matrix factorization (which can be seen as one type of graph embedding methods) have shown promising results, and hence there is a need to systematically evaluate the more recent graph embedding methods (e.g., random walk-based and neural network-based) in terms of their usability and potential to further the state-of-the-art. Results: We select 11 representative graph embedding methods and conduct a systematic comparison on three important biomedical link prediction tasks: drug-disease association (DDA) prediction, drugdrug interaction (DDI) prediction, protein-protein interaction (PPI) prediction, and two node classification tasks: medical term semantic type classification, protein function prediction. Our experimental results demonstrate that the recent graph embedding methods which have not been explored completely on biomedical tasks achieve promising results and deserve more attention in the future biomedical graph analysis. Besides, compared with three state-of-the-art methods for DDAs, DDIs and protein function predictions, the recent graph embedding methods achieve competitive performance without using any biological features and the learned embeddings can be treated as complementary representations for the biological features. By summarizing the experimental results, we provide general guidelines for properly selecting graph embedding methods and setting their hyper-parameters for different biomedical tasks. Availability: As part of our contributions in the paper, we develop an easy-to-use Python package with detailed instructions, BioNEV, available at: https://github.com/xiangyue9607/BioNEV, including all source code and datasets, to facilitate studying various graph embedding methods on biomedical tasks.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Node Classificationmentioning

confidence: 99%

See 2 more Smart Citations

Graph Embedding on Biomedical Networks: Methods, Applications, and Evaluations

Yue,

Wang,

Huang

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…(Finlayson et al, 2014) learned the UMLS CUIs embedding using 20 million clinical notes spanning 19 years of data from Stanford Hospital and Clinics using co-occurrence based analyses 12 . (Beam et al, 2018) also learned the UMLS CUIs embeddings, cui2vec, from medical billing codes, biomedical journal texts, and the clinical concept co-occurence matrix used in (Finlayson et al, 2014) 13 . (Choi et al, 2016e) learned three dense, low-dimensional embedding spaces of UMLS CUIs and billing codes from UMLS-processed journal abstracts, UMLS-processed clinical notes and claims data using the word2vec skip-gram framework 14 .…”

Section: Concept Representationsmentioning

confidence: 99%

Representation Learning for Electronic Health Records

Weng,

Szolovits

2019

Preprint

View full text Add to dashboard Cite

Information in electronic health records (EHR), such as clinical narratives, examination reports, lab measurements, demographics, and other patient encounter entries, can be transformed into appropriate data representations that can be used for downstream clinical machine learning tasks using representation learning. Learning better representations is critical to improve the performance of downstream tasks. Due to the advances in machine learning, we now can learn better and meaningful representations from EHR through disentangling the underlying factors inside data and distilling large amounts of information and knowledge from heterogeneous EHR sources.

show abstract

Spherical Regression Under Mismatch Corruption With Application to Automated Knowledge Translation

Shi

Cai

2020

Journal of the American Statistical Association

Self Cite

View full text Add to dashboard Cite

Motivated by a series of applications in data integration, language translation, bioinformatics, and computer vision, we consider spherical regression with two sets of unit-length vectors when the data are corrupted by a small fraction of mismatch in the response-predictor pairs. We propose a three-step algorithm in which we initialize the parameters by solving an orthogonal Procrustes problem to estimate a translation matrix W ignoring the mismatch. We then estimate a mapping matrix aiming to correct the mismatch using hard-thresholding to induce sparsity, while incorporating potential group information. We eventually obtain a refined estimate for W by removing the estimated mismatched pairs. We derive the error bound for the initial estimate of W in both fixed and high-dimensional setting. We demonstrate that the refined estimate of W achieves an error rate that is as good as if no mismatch is present. We show that our mapping recovery method not only correctly distinguishes one-to-one and one-to-many correspondences, but also consistently identifies the matched pairs and estimates the weight vector for combined correspondence. We examine the finite sample performance of the proposed method via extensive simulation studies, and with application to the unsupervised translation of medical codes using electronic health records data.

show abstract

Clinical Concept Embeddings Learned from Massive Sources of Multimodal Medical Data

Cited by 10 publications

References 0 publications

Graph Embedding on Biomedical Networks: Methods, Applications, and Evaluations

Graph Embedding on Biomedical Networks: Methods, Applications, and Evaluations

Representation Learning for Electronic Health Records

Spherical Regression Under Mismatch Corruption With Application to Automated Knowledge Translation

Contact Info

Product

Resources

About