Evaluation of knowledge graph embedding approaches for drug-drug interaction prediction in realistic settings

Abstract

BackgroundCurrent approaches to identifying drug-drug interactions (DDIs), include safety studies during drug development and post-marketing surveillance after approval, offer important opportunities to identify potential safety issues, but are unable to provide complete set of all possible DDIs. Thus, the drug discovery researchers and healthcare professionals might not be fully aware of potentially dangerous DDIs. Predicting potential drug-drug interaction helps reduce unanticipated drug interactions and drug development costs and optimizes the drug design process. Methods for prediction of DDIs have the tendency to report high accuracy but still have little impact on translational research due to systematic biases induced by networked/paired data. In this work, we aimed to present realistic evaluation settings to predict DDIs using knowledge graph embeddings. We propose a simple disjoint cross-validation scheme to evaluate drug-drug interaction predictions for the scenarios where the drugs have no known DDIs.ResultsWe designed different evaluation settings to accurately assess the performance for predicting DDIs. The settings for disjoint cross-validation produced lower performance scores, as expected, but still were good at predicting the drug interactions. We have applied Logistic Regression, Naive Bayes and Random Forest on DrugBank knowledge graph with the 10-fold traditional cross validation using RDF2Vec, TransE and TransD. RDF2Vec with Skip-Gram generally surpasses other embedding methods. We also tested RDF2Vec on various drug knowledge graphs such as DrugBank, PharmGKB and KEGG to predict unknown drug-drug interactions. The performance was not enhanced significantly when an integrated knowledge graph including these three datasets was used.ConclusionWe showed that the knowledge embeddings are powerful predictors and comparable to current state-of-the-art methods for inferring new DDIs. We addressed the evaluation biases by introducing drug-wise and pairwise disjoint test classes. Although the performance scores for drug-wise and pairwise disjoint seem to be low, the results can be considered to be realistic in predicting the interactions for drugs with limited interaction information.

Drug-Drug Interaction Prediction Based on Knowledge Graph Embeddings and Convolutional-LSTM Network

“…For the experiment, 80% of the data is used for the training using 5-fold cross-validation and evaluate the optimized model on 20% held-out data in which the best hyperparameters were produced through a random search. Although AUC score is used commonly as a performance metric in previous studies, literature has emphasized that it might not be sufficiently accurate for imbalanced data [8,24]. Therefore, we used the area under the precision-recall curve (AUPR), and Matthias correlation coefficient (MCC) along with the AUC and F1-score to measures the performance of the classifiers.…”

“…Traditional work relies on in vitro and in vivo experiments and focuses on small sets of specific drug pairs and had laboratory limitations [15]. With the emergence of available biomedical data, researchers moved the focus towards automatically populating and completing biomedical KGs using large-scale structured databases and text publicly available [8]. In this scope, the Bio2RDF project made 35 life sciences datasets as linked open data (LOD) in RDF, in which similar entities are mapped in different KGs and built large heterogeneous graphs that also contain biomedical drug-related facts.…”

“…In this scope, the Bio2RDF project made 35 life sciences datasets as linked open data (LOD) in RDF, in which similar entities are mapped in different KGs and built large heterogeneous graphs that also contain biomedical drug-related facts. Although these approaches made available numerous biomedical KGs, they often contain incomplete and inaccurate data that impede their application in the field of safe medicine development [8]. Lately, ML and text mining based approaches were used in which pharmacological similarities of drugs as features are used by regarding the DDIs prediction task as a link prediction problem.…”

“…However, feature-based approaches only predict binary DDIs or those that have been pre-defined in structured databases and suffer from robustness caused by data sparsity and vast computation requirements. Similarity-based approaches, in contrast, do not allow for the calculation of various similarities for many drugs due to lack of drug information [8].…”

“…By combining these two deep architectures, the convolutional-LSTM(Conv-LSTM) can capture both locally and globally important drug features which we found to lead to more accurate DDI predictions [37]. However, the features which have been traditionally used for these approaches form either a large and sparse binary matrix or a dense, but small similarity matrix, making them not ideal for training ML models [8]. Further, an increasing amount of drug and small molecules data is being generated, and state-of-the-art approaches still rely on the analysis on a limited number of data sources only, e.g.…”