Adapt-Kcr: a novel deep learning framework for accurate prediction of lysine crotonylation sites based on learning embedding features and attention architecture

Abstract

Protein lysine crotonylation (Kcr) is an important type of posttranslational modification that is associated with a wide range of biological processes. The identification of Kcr sites is critical to better understanding their functional mechanisms. However, the existing experimental techniques for detecting Kcr sites are cost-ineffective, to a great need for new computational methods to address this problem. We here describe Adapt-Kcr, an advanced deep learning model that utilizes adaptive embedding and is based on a convolutional neural network together with a bidirectional long short-term memory network and attention architecture. On the independent testing set, Adapt-Kcr outperformed the current state-of-the-art Kcr prediction model, with an improvement of 3.2% in accuracy and 1.9% in the area under the receiver operating characteristic curve. Compared to other Kcr models, Adapt-Kcr additionally had a more robust ability to distinguish between crotonylation and other lysine modifications. Another model (Adapt-ST) was trained to predict phosphorylation sites in SARS-CoV-2, and outperformed the equivalent state-of-the-art phosphorylation site prediction model. These results indicate that self-adaptive embedding features perform better than handcrafted features in capturing discriminative information; when used in attention architecture, this could be an effective way of identifying protein Kcr sites. Together, our Adapt framework (including learning embedding features and attention architecture) has a strong potential for prediction of other protein posttranslational modification sites.

BERT-PPII: The Polyproline Type II Helix Structure Prediction Model Based on BERT and Multichannel CNN

“…This section conducts the comparative experiments on the balanced dataset and the comparative methods including ANN [ 25 ], SVM [ 26 ], random forest (RF), K-Nearest Neighbor (KNN), FAD-BERT [ 19 ], EECL [ 10 ], Adapt_Kcr [ 40 ], and BERT4Bitter [ 20 ]. All comparative methods use one-hot to encode the amino acid residues.…”

“…In this paper, we compare BERT-PPII method with the following methods. To predict PPII helices on a balanced dataset, Siermala et al ANN [25], SVM [26], random forest (RF), K-Nearest Neighbor (KNN), FAD-BERT [19], EECL [10], Adapt_Kcr [40], and BERT4Bitter [20]. All comparative methods use onehot to encode the amino acid residues.…”

PD-BertEDL: An Ensemble Deep Learning Method Using BERT and Multivariate Representation to Predict Peptide Detectability

“…In the BERT model, the input vector of each word consists of three embeddings: token embedding, segment embedding and position embedding. Position embedding can represent the absolute position information of each word in a sentence [ 47 ]. A visualization of the word embedding structure is shown in Figure 5 .…”

“…In order to fully capture the context information of the peptide, we regarded the peptide sequence as a sentence, and took k amino acids as a group, which was called “word” [ 47 ]. Since the parameter scale of BERT model is up to 100 million, it is very demanding for the experimental environment to use our own corpus to build a word list and retrain the BERT model.…”

Antimicrobial Peptides Prediction method based on sequence multidimensional feature embedding

“…Deep learning methods can automatically learn features from the raw data through convolution operations, avoiding the loss of data features. Various deep learning methods have been applied in protein sequence classification, such as bidirectional long short-term memory network (Bi-LSTM) ( Tng et al, 2021 ; Zhang Y. et al, 2022 ; Zhang et al, 2022c ; Li et al, 2022 ; Qiao et al, 2022 ; Wang et al, 2022 ), two-dimensional convolutional neural network (2D CNN) ( Le et al, 2021 ), deep residual network (ResNet) ( Xu et al, 2021 ), graph convolutional network (GCN) ( Chen et al, 2021 ), deep neural network (DNN) ( Gao et al, 2019 ; Han et al, 2019 ; Le et al, 2019 ; Hathaway et al, 2021 ), and Recurrent Neural Network (RNN) ( Zheng et al, 2020 ; Yun et al, 2021 ). These research methods have generally achieved good classification results and have attracted increasing attention.…”