Applying deep learning techniques on medical corpora from the World Wide Web: a prototypical system and evaluation

Miñarro-Giménez, José Antonio; Marín-Alonso, Oscar; Samwald, Matthias

doi:10.48550/arxiv.1502.03682

Cited by 3 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In previous reports, Word2Vec, specifically the skipgram architecture, achieved the highest score on three of four rated tasks: analogy-based operations, odd one similarity, and human validation [18]. Skip-grams also performed better in biomedical studies [19][20][21][22].…”

Section: Model Learningmentioning

confidence: 96%

Validating the representation of distance between infarct diseases using word embedding

Yokokawa

Noda

Yanagita

et al. 2022

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

Background The pivot and cluster strategy (PCS) is a diagnostic reasoning strategy that automatically elicits disease clusters similar to a differential diagnosis in a batch. Although physicians know empirically which disease clusters are similar, there has been no quantitative evaluation. This study aimed to determine whether inter-disease distances between word embedding vectors using the PCS are a valid quantitative representation of similar disease groups in a limited domain. Methods Abstracts were extracted from the Ichushi Web database and subjected to morphological analysis and training using Word2Vec, FastText, and GloVe. Consequently, word embedding vectors were obtained. For words including “infarction,” we calculated the cophenetic correlation coefficient (CCC) as an internal validity measure and the adjusted rand index (ARI), normalized mutual information (NMI), and adjusted mutual information (AMI) with ICD-10 codes as the external validity measures. This was performed for each combination of metric and hierarchical clustering method. Results Seventy-one words included “infarction,” of which 38 diseases matched the ICD-10 standard with the appearance of 21 unique ICD-10 codes. When using Word2Vec, the CCC was most significant at 0.8690 (metric and method: euclidean and centroid), whereas the AMI was maximal at 0.4109 (metric and method: cosine and correlation, and average and weighted). The NMI and ARI were maximal at 0.8463 and 0.3593, respectively (metric and method: cosine and complete). FastText and GloVe generally resulted in the same trend as Word2Vec, and the metric and method that maximized CCC differed from the ones that maximized the external validity measures. Conclusions The metric and method that maximized the internal validity measure differed from those that maximized the external validity measures; both produced different results. The cosine distance should be used when considering ICD-10, and the Euclidean distance when considering the frequency of word occurrence. The distributed representation, when trained by Word2Vec on the “infarction” domain from a Japanese academic corpus, provides an objective inter-disease distance used in PCS.

show abstract

Section: Model Learningmentioning

confidence: 96%

Validating the representation of distance between infarct diseases using word embedding

Yokokawa

Noda

Yanagita

et al. 2022

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

show abstract

“…This kind of method has poor scalability. Following their introduction, the use of pretrained language models, such as Word2vec [6], ELMo [7], BERT [8], RoBERTa [9], and XLNET [10], to complete Knowledge Graphs [11,12], including Medical Knowledge Graphs [13][14][15][16], has become a popular research topic. The literature [11] proposes a method based on XLNET and a classification model to verify whether the triples of a Knowledge Graph are valid for relation completion.…”

Section: Related Workmentioning

confidence: 99%

“…Similar to [11], this method also performs relationship completion through two classifications. Another study [13] only applies Word2vec to the identification of relationships from unstructured text. However, compared with other published results, the results of this method are very limited.…”

Section: Related Workmentioning

confidence: 99%

“…Most of the early methods for MKGC, such as [5], are based on rules. Following their introduction, the use of pretrained language models, such as Word2vec [6], ELMo [7], BERT [8], RoBERTa [9], and XLNET [10], to complete Knowledge Graphs [11,12], including Medical Knowledge Graphs [13][14][15][16], has become a popular research topic. The biggest limitation of these existing works is that they focus on completing relationships or classes, and rarely involve directly obtaining entities and RDF triples from unstructured text.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Medical Knowledge Graph Completion Based on Word Embeddings

Gao

Lü

Chen³

2022

Information

View full text Add to dashboard Cite

The aim of Medical Knowledge Graph Completion is to automatically predict one of three parts (head entity, relationship, and tail entity) in RDF triples from medical data, mainly text data. Following their introduction, the use of pretrained language models, such as Word2vec, BERT, and XLNET, to complete Medical Knowledge Graphs has become a popular research topic. The existing work focuses mainly on relationship completion and has rarely solved entities and related triples. In this paper, a framework to predict RDF triples for Medical Knowledge Graphs based on word embeddings (named PTMKG-WE) is proposed, for the specific use for the completion of entities and triples. The framework first formalizes existing samples for a given relationship from the Medical Knowledge Graph as prior knowledge. Second, it trains word embeddings from big medical data according to prior knowledge through Word2vec. Third, it can acquire candidate triples from word embeddings based on analogies from existing samples. In this framework, the paper proposes two strategies to improve the relation features. One is used to refine the relational semantics by clustering existing triple samples. Another is used to accurately embed the expression of the relationship through means of existing samples. These two strategies can be used separately (called PTMKG-WE-C and PTMKG-WE-M, respectively), and can also be superimposed (called PTMKG-WE-C-M) in the framework. Finally, in the current study, PubMed data and the National Drug File-Reference Terminology (NDF-RT) were collected, and a series of experiments was conducted. The experimental results show that the framework proposed in this paper and the two improvement strategies can be used to predict new triples for Medical Knowledge Graphs, when medical data are sufficiently abundant and the Knowledge Graph has appropriate prior knowledge. The two strategies designed to improve the relation features have a significant effect on the lifting precision, and the superposition effect becomes more obvious. Another conclusion is that, under the same parameter setting, the semantic precision of word embedding can be improved by extending the breadth and depth of data, and the precision of the prediction framework in this paper can be further improved in most cases. Thus, collecting and training big medical data is a viable method to learn more useful knowledge.

show abstract

“…Skip-grams also performed better in biomedical studies. [17,18,19,20] This study used a skip-gram algorithm based on previous studies, with a vector of 200 dimensions. We set the window size as 5, minimum count as 5, and the number of iterations as 100.…”

Section: Model Learningmentioning

confidence: 99%

Validating the representation of distance between infarct diseases using Word2Vec word embedding

Yokokawa

Noda

Yanagita

et al. 2022

Preprint

View full text Add to dashboard Cite

Objective: To determine if inter-disease distances between word embedding vectors using the picot-and-cluster strategy (PCS) are a valid quantitative representation of similar disease groups in a limited domain. Materials and Methods: Abstracts were extracted from the Ichushi-Web database and subjected to morphological analysis and training using the Word2Vec. From this, word embedding vectors were obtained. For words including "infarction", we calculated the cophenetic correlation coefficient (CCC) as an internal validity measure and the adjusted rand index (ARI), normalized mutual information (NMI), and adjusted mutual information (AMI) with ICD-10 codes as the external validity measures. This was performed for each combination of metric and hierarchical clustering method. Results: Seventy-one words included "infarction", of which 38 diseases matched the ICD-10 standard with the appearance of 21 unique ICD-10 codes. The CCC was most significant at 0.8690 (metric and method: euclidean and centroid), while the AMI was maximal at 0.4109 (metric and method: cosine and correlation, and average and weighted). The NMI and ARI were maximal at 0.8463 and 0.3593, respectively (metric and method: cosine and complete). Discussion: The metric and method that maximized the internal validity measure were different from those that maximized the external validity measures; both produced different results. The Cosine distance should be used when considering ICD-10, and the Euclidean distance when considering the frequency of word occurrence. Conclusion: The distributed representation, when trained by Word2Vec on the "infarction" domain from a Japanese academic corpus, provides an objective inter-disease distance used in PCS.

show abstract

Applying deep learning techniques on medical corpora from the World Wide Web: a prototypical system and evaluation

Cited by 3 publications

References 0 publications

Validating the representation of distance between infarct diseases using word embedding

Validating the representation of distance between infarct diseases using word embedding

Medical Knowledge Graph Completion Based on Word Embeddings

Validating the representation of distance between infarct diseases using Word2Vec word embedding

Contact Info

Product

Resources

About