CascadER: Cross-Modal Cascading for Knowledge Graph Link Prediction

Safavi, Tara; Downey, Doug; Hope, Tom

doi:10.48550/arxiv.2205.08012

Cited by 3 publications

(2 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to the recency of [37], we leave additional comparisons beyond our real GPT2 baseline to future work. Applying language models to knowledge graphs has been investigated in the general [91,30,93] and scientific domains [49,63]. They can be considered similar to our tests of BERT language models applied to a drug synergy hypergraph ( § 4.1).…”

Section: Language Models For Chemistry and Knowledge Graph Completionmentioning

confidence: 99%

SynerGPT: In-Context Learning for Personalized Drug Synergy Prediction and Drug Design

Edwards

Naik

Khot

et al. 2023

Preprint

View full text Add to dashboard Cite

Predicting synergistic drug combinations can help accelerate discovery of cancer treatments, particularly therapies personalized to a patient's specific tumor via biopsied cells. In this paper, we propose a novel setting and models for in-context drug synergy learning. We are given a small "personalized dataset" of 10-20 drug synergy relationships in the context of specific cancer cell targets. Our goal is to predict additional drug synergy relationships in that context. Inspired by recent work that pre-trains a GPT language model (LM) to "in-context learn" common function classes, we devise novel pre-training schemes that enable a GPT model to in-context learn "drug synergy functions". Our model—which does not use any textual corpora, molecular fingerprints, protein interaction or any other domain-specific knowledge— is able to achieve competitive results. We further integrate our in-context approach with a genetic algorithm to optimize model prompts and select synergy candidates to test after conducting a patient biopsy. Finally, we explore a novel task of inverse drug design which can potentially enable the design of drugs that synergize specifically to target a given patient's "personalized dataset". Our findings can potentially have an important impact on precision cancer medicine, and also raise intriguing questions on non-textual pre-training for LMs.

show abstract

Section: Language Models For Chemistry and Knowledge Graph Completionmentioning

confidence: 99%

SynerGPT: In-Context Learning for Personalized Drug Synergy Prediction and Drug Design

Edwards

Naik

Khot

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…two approaches are completely different learning paradigms, it is difficult to fully utilize both types of features within a single learning model, especially deep features 14 . These issues prevent fully leveraging the combination of structural and semantic features, which is crucial for effective prediction.…”

mentioning

confidence: 99%

A novel model for relation prediction in knowledge graphs exploiting semantic and structural feature integration

Yang,

Lu,

et al. 2024

Sci Rep

View full text Add to dashboard Cite

Relation prediction is a critical task in knowledge graph completion and associated downstream tasks that rely on knowledge representation. Previous studies indicate that both structural features and semantic information are meaningful for predicting missing relations in knowledge graphs. This has led to the development of two types of methods: structure-based methods and semantics-based methods. Since these two approaches represent two distinct learning paradigms, it is difficult to fully utilize both sets of features within a single learning model, especially deep features. As a result, existing studies usually focus on only one type of feature. This leads to an insufficient representation of knowledge in current methods and makes them prone to overlooking certain patterns when predicting missing relations. In this study, we introduce a novel model, RP-ISS, which combines deep semantic and structural features for relation prediction. The RP-ISS model utilizes a two-part architecture, with the first component being a RoBERTa module that is responsible for extracting semantic features from entity nodes. The second part of the system employs an edge-based relational message-passing network designed to capture and interpret structural information within the data. To alleviate the computational burden of the message-passing network on the RoBERTa module during the sampling process, RP-ISS introduces a node embedding memory bank, which updates asynchronously to circumvent excessive computation. The model was assessed on three publicly accessible datasets (WN18RR, WN18, and FB15k-237), and the results revealed that RP-ISS surpasses all baseline methods across all evaluation metrics. Moreover, RP-ISS showcases robust performance in graph inductive learning.

show abstract

BioBLP: a modular framework for learning on multimodal biomedical knowledge graphs

Daza,

Alivanistos,

Mitra

et al. 2023

J Biomed Semant

View full text Add to dashboard Cite

Background Knowledge graphs (KGs) are an important tool for representing complex relationships between entities in the biomedical domain. Several methods have been proposed for learning embeddings that can be used to predict new links in such graphs. Some methods ignore valuable attribute data associated with entities in biomedical KGs, such as protein sequences, or molecular graphs. Other works incorporate such data, but assume that entities can be represented with the same data modality. This is not always the case for biomedical KGs, where entities exhibit heterogeneous modalities that are central to their representation in the subject domain. Objective We aim to understand how to incorporate multimodal data into biomedical KG embeddings, and analyze the resulting performance in comparison with traditional methods. We propose a modular framework for learning embeddings in KGs with entity attributes, that allows encoding attribute data of different modalities while also supporting entities with missing attributes. We additionally propose an efficient pretraining strategy for reducing the required training runtime. We train models using a biomedical KG containing approximately 2 million triples, and evaluate the performance of the resulting entity embeddings on the tasks of link prediction, and drug-protein interaction prediction, comparing against methods that do not take attribute data into account. Results In the standard link prediction evaluation, the proposed method results in competitive, yet lower performance than baselines that do not use attribute data. When evaluated in the task of drug-protein interaction prediction, the method compares favorably with the baselines. Further analyses show that incorporating attribute data does outperform baselines over entities below a certain node degree, comprising approximately 75% of the diseases in the graph. We also observe that optimizing attribute encoders is a challenging task that increases optimization costs. Our proposed pretraining strategy yields significantly higher performance while reducing the required training runtime. Conclusion BioBLP allows to investigate different ways of incorporating multimodal biomedical data for learning representations in KGs. With a particular implementation, we find that incorporating attribute data does not consistently outperform baselines, but improvements are obtained on a comparatively large subset of entities below a specific node-degree. Our results indicate a potential for improved performance in scientific discovery tasks where understudied areas of the KG would benefit from link prediction methods.

show abstract

CascadER: Cross-Modal Cascading for Knowledge Graph Link Prediction

Cited by 3 publications

References 28 publications

SynerGPT: In-Context Learning for Personalized Drug Synergy Prediction and Drug Design

SynerGPT: In-Context Learning for Personalized Drug Synergy Prediction and Drug Design

A novel model for relation prediction in knowledge graphs exploiting semantic and structural feature integration

BioBLP: a modular framework for learning on multimodal biomedical knowledge graphs

Contact Info

Product

Resources

About