GeneralizedDTA: combining pre-training and multi-task learning to predict drug-target binding affinity for unknown drug discovery

Abstract

Background Accurately predicting drug-target binding affinity (DTA) in silico plays an important role in drug discovery. Most of the computational methods developed for predicting DTA use machine learning models, especially deep neural networks, and depend on large-scale labelled data. However, it is difficult to learn enough feature representation from tens of millions of compounds and hundreds of thousands of proteins only based on relatively limited labelled drug-target data. There are a large number of unknown drugs, which never appear in the labelled drug-target data. This is a kind of out-of-distribution problems in bio-medicine. Some recent studies adopted self-supervised pre-training tasks to learn structural information of amino acid sequences for enhancing the feature representation of proteins. However, the task gap between pre-training and DTA prediction brings the catastrophic forgetting problem, which hinders the full application of feature representation in DTA prediction and seriously affects the generalization capability of models for unknown drug discovery. Results To address these problems, we propose the GeneralizedDTA, which is a new DTA prediction model oriented to unknown drug discovery, by combining pre-training and multi-task learning. We introduce self-supervised protein and drug pre-training tasks to learn richer structural information from amino acid sequences of proteins and molecular graphs of drug compounds, in order to alleviate the problem of high variance caused by encoding based on deep neural networks and accelerate the convergence of prediction model on small-scale labelled data. We also develop a multi-task learning framework with a dual adaptation mechanism to narrow the task gap between pre-training and prediction for preventing overfitting and improving the generalization capability of DTA prediction model on unknown drug discovery. To validate the effectiveness of our model, we construct an unknown drug data set to simulate the scenario of unknown drug discovery. Compared with existing DTA prediction models, the experimental results show that our model has the higher generalization capability in the DTA prediction of unknown drugs. Conclusions The advantages of our model are mainly attributed to two kinds of pre-training tasks and the multi-task learning framework, which can learn richer structural information of proteins and drugs from large-scale unlabeled data, and then effectively integrate it into the downstream prediction task for obtaining a high-quality DTA prediction in unknown drug discovery.

Machine Learning for Sequence and Structure-Based Protein–Ligand Interaction Prediction

“…To solve the problem of scarce labeling data, transfer learning is applied to encode the protein characteristics of the pretrained converter. Moreover, GeneralizedDTA 50 introduces self-supervised protein and drug pretraining tasks to learn more structural information from the amino acid protein sequences and molecular diagrams. Furthermore, DrugAI 51 proposed a multiview DL model that combines four modules (e.g., a CNN for targets, a network embedding module for drugs and targets, and a deep neural network (DNN)) for predicting binary interactions between drugs and targets.…”

AI Approaches for the Discovery and Validation of Drug Targets

“…Within their PUL framework, Raies et al It is also possible to combine multiple data modalities in a more direct way than ensemble modelling, namely via multitask learning (Caruana 1998). A multitask learning problem in drug target discovery is typically framed as one where you are trying to predict target qualities as well as properties of the target-binding drug (Lin et al 2022;Sadawi et al 2019). Multitask learning allows the model to co-learn a set of tasks together to optimise overall performance.…”

A comprehensive review of the recent advances on predicting drug-target affinity based on deep learning

“…Hybrid-based methods (Karimi et al, 2021;Wang et al, 2021b;Zhang et al, 2021;Cheng et al, 2022;Li et al, 2022a;Lin et al, 2022a;Tian et al, 2022;Yang et al, 2022;Jiang et al, 2023;Pan et al, 2023;Wang et al, 2023a;Wang and Li, 2023;Xia et al, 2023;Yang et al, 2023;Zeng et al, 2023;Zhang et al, 2023b;Zhang et al, 2023a;Zhu et al, 2023a;Zhu et al, 2023c;Nguyen et al, 2022.) leverage deep learning models to extract sequence features from drug SMILES and target sequences, as well as the structural features from twodimensional molecular topology graphs and three-dimensional structures of drug small molecules. These methods focus on integrating the structural features of drugs into sequence-based approaches.…”