FusionDTA: attention-based feature polymerizer and knowledge distillation for drug-target binding affinity prediction

Abstract

The prediction of drug-target affinity (DTA) plays an increasingly important role in drug discovery. Nowadays, lots of prediction methods focus on feature encoding of drugs and proteins, but ignore the importance of feature aggregation. However, the increasingly complex encoder networks lead to the loss of implicit information and excessive model size. To this end, we propose a deep-learning-based approach namely FusionDTA. For the loss of implicit information, a novel muti-head linear attention mechanism was utilized to replace the rough pooling method. This allows FusionDTA aggregates global information based on attention weights, instead of selecting the largest one as max-pooling does. To solve the redundancy issue of parameters, we applied knowledge distillation in FusionDTA by transfering learnable information from teacher model to student. Results show that FusionDTA performs better than existing models for the test domain on all evaluation metrics. We obtained concordance index (CI) index of 0.913 and 0.906 in Davis and KIBA dataset respectively, compared with 0.893 and 0.891 of previous state-of-art model. Under the cold-start constrain, our model proved to be more robust and more effective with unseen inputs than baseline methods. In addition, the knowledge distillation did save half of the parameters of the model, with only 0.006 reduction in CI index. Even FusionDTA with half the parameters could easily exceed the baseline on all metrics. In general, our model has superior performance and improves the effect of drug–target interaction (DTI) prediction. The visualization of DTI can effectively help predict the binding region of proteins during structure-based drug design.

Geometric Interaction Graph Neural Network for Predicting Protein–Ligand Binding Affinities from 3D Structures (GIGN)

“…Interaction-free methods ,− ,− ,− implicitly assume that ML models can predict PLA from data that do not reveal physical protein–ligand interactions. Therefore, ligands are often expressed by the simplified molecular-input line-entry system (SMILES), or 2D graphs, and proteins are represented by sequences, while atomic interactions are omitted for simplicity.…”

BiComp-DTA: Drug-target binding affinity prediction through complementary biological-related and compression-based featurization approach

“…To evaluate the method’s robustness and generalization, especially for unseen data, we conducted widely-used alternative splitting data settings, named cold-start settings [ 24 , 44 ]. For this purpose, three settings have been applied for training and testing the method, including cold-protein, cold-drug, and cold-drug-protein for which, the model testing is performed for unseen protein, unseen drug, and unseen drug-protein pairs in the training set, respectively.…”

“…Despite generating informative distributed representation vectors, the network architecture suffers from the overall complexity. Recently, FusionDTA [ 24 ] adopted ESM-1b [ 25 ] transformer for generating distribution representation vectors from the protein sequences. Although this method has shown promising performance, it relies on extra pre-training and fine-tuning stages for efficient protein sequence encoding.…”

Mitigating cold-start problems in drug-target affinity prediction with interaction knowledge transferring

“…In the DTA task, we test our proposed method in cold-start settings, including cold-drug and cold-target. We follow previous works [ 1 , 47 ] on cold start splitting process. In the cold drug setting, all drugs in the validation and test set are absent from the training set.…”