TransformerCPI: improving compound–protein interaction prediction by sequence-based deep learning with self-attention mechanism and label reversal experiments

Abstract

Motivation Identifying compound-protein interaction (CPI) is a crucial task in drug discovery and chemogenomics studies, and proteins without three-dimensional (3D) structure account for a large part of potential biological targets, which requires developing methods using only protein sequence information to predict CPI. However, sequence-based CPI models may face some specific pitfalls, including using inappropriate datasets, hidden ligand bias, and splitting datasets inappropriately, resulting in overestimation of their prediction performance. Results To address these issues, we here constructed new datasets specific for CPI prediction, proposed a novel transformer neural network named TransformerCPI, and introduced a more rigorous label reversal experiment to test whether a model learns true interaction features. TransformerCPI achieved much improved performance on the new experiments, and it can be deconvolved to highlight important interacting regions of protein sequences and compound atoms, which may contribute chemical biology studies with useful guidance for further ligand structural optimization. Supplementary information Supplementary data are available at Bioinformatics online. Availability and implementation https://github.com/lifanchen-simm/transformerCPI

Deep Learning in Proteomics

“…Recent studies show that models based on natural language processing inspired techniques such as Transformer, [217] BERT, [218] and GPT-2 [219] can learn features from a large corpus of protein sequences in a self-supervised fashion, with applications in a variety of downstream tasks. [220,221] Besides a linear sequence of amino acids, proteins can also be modeled as a graph to capture both structure and sequence information. Graph neural networks [222] are powerful deep learning architectures for learning representations of nodes and edges from such data.…”

A deep learning framework for elucidating whole-genome chemical interaction space

“…Its ROC-AUC and PR-AUC is 0.476 and 0.261, respectively. The Transformer-based architecture has achieved the state-of-the-art performance in the recently published work [16]. Indeed, it outperforms LSTM model with a ROC-AUC of 0.648 and a PR-AUC of 0.380, respectively.…”

“…End-to-end deep learning approaches have recently gained a momentum [10] [11]. Various neural network architectures, including Convolutional Neural Network (CNN) [12] [13], seq2seq [14] [15], and Transformer [16], have been applied to represent protein sequences. These works mainly focused on filling in missing CPIs for the existing drug targets.…”

MSA-Regularized Protein Sequence Transformer toward Predicting Genome-Wide Chemical-Protein Interactions: Application to GPCRome Deorphanization

“…End-to-end deep learning approaches have recently gained momentum. 10 11 Various neural network architectures, including Convolutional Neural Network (CNN), 12 13 seq2seq, 14 15 and Transformer, 16 have been applied to represent protein sequences. These works mainly focused on filling in missing CPIs for the existing drug targets.…”