Embedding Biomedical Ontologies by Jointly Encoding Network Structure and Textual Node Descriptors

Kotitsas, Sotiris; Pappas, Dimitris; Androutsopoulos, Ion; McDonald, Ryan; Apidianaki, Marianna

doi:10.18653/v1/w19-5032

Cited by 15 publications

(11 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…NE methods such as Node2Vec (Grover and Leskovec, 2016) learn embeddings for nodes in a network (graph) by applying a variant of the skipgram model on samples generated using random walks, and they have shown impressive results on node classification and link prediction tasks on a wide range of network datasets. In the biomedical domain, CANode2Vec (Kotitsas et al, 2019) applied several NE methods on single-relation subsets of the SNOMED-CT graph, but the lack of comparison to existing methods and the disregard for the heterogeneous structure of the knowledge graph substantially limit its significance.…”

Section: Biomedical Concept Embeddingsmentioning

confidence: 99%

Benchmark and Best Practices for Biomedical Knowledge Graph Embeddings

Balažević

Allen

Chawla

et al. 2020

Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing

View full text Add to dashboard Cite

Much of biomedical and healthcare data is encoded in discrete, symbolic form such as text and medical codes. There is a wealth of expert-curated biomedical domain knowledge stored in knowledge bases and ontologies, but the lack of reliable methods for learning knowledge representation has limited their usefulness in machine learning applications. While text-based representation learning has significantly improved in recent years through advances in natural language processing, attempts to learn biomedical concept embeddings so far have been lacking. A recent family of models called knowledge graph embeddings have shown promising results on general domain knowledge graphs, and we explore their capabilities in the biomedical domain. We train several state-of-the-art knowledge graph embedding models on the SNOMED-CT knowledge graph, provide a benchmark with comparison to existing methods and indepth discussion on best practices, and make a case for the importance of leveraging the multi-relational nature of knowledge graphs for learning biomedical knowledge representation. The embeddings, code, and materials will be made available to the community 1 .

show abstract

Section: Biomedical Concept Embeddingsmentioning

confidence: 99%

Benchmark and Best Practices for Biomedical Knowledge Graph Embeddings

Balažević

Allen

Chawla

et al. 2020

Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing

View full text Add to dashboard Cite

show abstract

“…The observation that nearby entities are more semantically similar (Figure 2b) motivates us to integrate textual similarity with graph topological similarity to boost the entity normalization. Conventional approaches often integrate text and graph information by adapting a graph-based framework and incorporating text features as node features (Kotitsas et al, 2019). However, such approaches might not fully utilize the strong generalization ability of pre-trained models, which have been crucial for a variety of NLP tasks (Devlin et al, 2018;Petroni et al, 2019).…”

Section: Intuitionmentioning

confidence: 99%

GraphPrompt: Biomedical Entity Normalization Using Graph-based Prompt Templates

Zhang

Wang

Zhang

et al. 2021

Preprint

View full text Add to dashboard Cite

Biomedical entity normalization unifies the language across biomedical experiments and studies, and further enables us to obtain a holistic view of life sciences. Current approaches mainly study the normalization of more standardized entities such as diseases and drugs, while disregarding the more ambiguous but crucial entities such as pathways, functions and cell types, hindering their real-world applications. To achieve biomedical entity normalization on these under-explored entities, we first introduce an expert-curated dataset OBO-syn encompassing 70 different types of entities and 2 million curated entity-synonym pairs. To utilize the unique graph structure in this dataset, we propose GraphPrompt, a promptbased learning approach that creates prompt templates according to the graphs. Graph-Prompt obtained 41.0% and 29.9% improvement on zero-shot and few-shot settings respectively, indicating the effectiveness of these graph-based prompt templates. We envision that our method GraphPrompt and OBO-syn dataset can be broadly applied to graph-based NLP tasks, and serve as the basis for analyzing diverse and accumulating biomedical data.

show abstract

“…As an alternative approach to exploit the label hierarchy, we used a recent improvement of NODE2VEC (Grover and Leskovec, 2016) by Kotitsas et al (2019) to obtain alternative hierarchy-aware label representations. NODE2VEC is similar to WORD2VEC (Mikolov et al, 2013), but pre-trains node embeddings instead of word embeddings, replacing WORD2VEC's text windows by random walks on a graph (here the label hierarchy).…”

Section: Dn-bigru-lwanmentioning

confidence: 99%

An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels

Chalkidis¹,

Fergadiotis²,

Kotitsas³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Self Cite

View full text Add to dashboard Cite

Large-scale Multi-label Text Classification (LMTC) has a wide range of Natural Language Processing (NLP) applications and presents interesting challenges. First, not all labels are well represented in the training set, due to the very large label set and the skewed label distributions of LMTC datasets. Also, label hierarchies and differences in human labelling guidelines may affect graph-aware annotation proximity. Finally, the label hierarchies are periodically updated, requiring LMTC models capable of zero-shot generalization. Current state-of-the-art LMTC models employ Label-Wise Attention Networks (LWANs), which (1) typically treat LMTC as flat multi-label classification; (2) may use the label hierarchy to improve zero-shot learning, although this practice is vastly understudied; and (3) have not been combined with pre-trained Transformers (e.g. BERT), which have led to state-of-the-art results in several NLP benchmarks. Here, for the first time, we empirically evaluate a battery of LMTC methods from vanilla LWANs to hierarchical classification approaches and transfer learning, on frequent, few, and zero-shot learning on three datasets from different domains. We show that hierarchical methods based on Probabilistic Label Trees (PLTs) outperform LWANs. Furthermore, we show that Transformer-based approaches outperform the state-of-the-art in two of the datasets, and we propose a new state-of-the-art method which combines BERT with LWAN. Finally, we propose new models that leverage the label hierarchy to improve few and zero-shot learning, considering on each dataset a graph-aware annotation proximity measure that we introduce.

show abstract

Embedding Biomedical Ontologies by Jointly Encoding Network Structure and Textual Node Descriptors

Cited by 15 publications

References 28 publications

Benchmark and Best Practices for Biomedical Knowledge Graph Embeddings

Benchmark and Best Practices for Biomedical Knowledge Graph Embeddings

GraphPrompt: Biomedical Entity Normalization Using Graph-based Prompt Templates

An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels

Contact Info

Product

Resources

About