Time-resolved evaluation of compound repositioning predictions on a text-mined knowledge network

Abstract

BackgroundComputational compound repositioning has the potential for identifying new uses for existing drugs, and new algorithms and data source aggregation strategies provide ever-improving results via in silico metrics. However, even with these advances, the number of compounds successfully repositioned via computational screening remains low. New strategies for algorithm evaluation that more accurately reflect the repositioning potential of a compound could provide a better target for future optimizations.ResultsUsing a text-mined database, we applied a previously described network-based computational repositioning algorithm, yielding strong results via cross-validation, averaging 0.95 AUROC on test-set indications. However, to better approximate a real-world scenario, we built a time-resolved evaluation framework. At various time points, we built networks corresponding to prior knowledge for use as a training set, and then predicted on a test set comprised of indications that were subsequently described. This framework showed a marked reduction in performance, peaking in performance metrics with the 1985 network at an AUROC of .797. Examining performance reductions due to removal of specific types of relationships highlighted the importance of drug-drug and disease-disease similarity metrics. Using data from future timepoints, we demonstrate that further acquisition of these kinds of data may help improve computational results.ConclusionsEvaluating a repositioning algorithm using indications unknown to input network better tunes its ability to find emerging drug indications, rather than finding those which have been randomly withheld. Focusing efforts on improving algorithmic performance in a time-resolved paradigm may further improve computational repositioning predictions.

Hetnet connectivity search provides rapid insights into how two biomedical entities are related

“…Nonetheless, Hetionet v1.0 remains one of the most comprehensive and integrative networks that consolidates biomedical knowledge into a manageable number of node and edge types [33]. Other integrative resources, some still under development, include Wikidata [34], SemMedDB [35,36,37]. SPOKE, and RTX-KG2c [38].…”

Revitalizing Cancer Treatment: Exploring the Role of Drug Repurposing

“…Utilizing PubMed Identifier numbers (PMID), multiple time-resolved networks representing knowledge up to specific dates can be generated. These time-resolved networks can then be evaluated for computational repositioning by training on indications known during the network's time period and testing on indications approved after that period, simulating real-world conditions more closely [119].…”

Drug Repurposing using consilience of Knowledge Graph Completion methods

“…Common drug repositioning method have identified prospective candidates through exploiting drug-drug and disease-disease similarities in a knowledge graph (4; 5; 6). Alternative approaches have utilized graph traversal algorithms like shortest path and random walks, or path ranking algorithms like degree weighted path count to prioritize paths linking a drug and a disease (5; 7; 8; 9; 10; 11). More specific graph traversal methods have applied deep learning methods by formalizing drug repositioning as a reinforcement learning task for link prediction (12; 13).…”