SecProCT: In Silico Prediction of Human Secretory Proteins Based on Capsule Network and Transformer

Du, Wei; Zhao, Xuan; Sun, Yujiao; Zheng, Lei; Li, Ying

doi:10.3390/ijms22169054

Cited by 10 publications

(4 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hu et al developed a graph-based Transformer model to predict protein normal mode frequencies and demonstrated the superior capabilities of transformer models to predict protein properties using protein sequence data [19]. Du et al and Tang et al employed transformer models to predict human secretory proteins and identify plasmid contigs, respectively, exemplifying the versatility and potential of transformer models to advance our comprehension of protein sequences and structures across diverse contexts [20,21]. Most recently, Wan and Jiang (2023) developed TransCrispr, a hybrid Transformer and CNN model to predict cleavage efficiency for CRISPR-Cas9, and showed superior prediction accuracy and generalization ability compared to CNN and RNN-based methods.…”

Section: Related Workmentioning

confidence: 99%

Transformer-Based Deep Learning Model with Latent Space Regularization for CRISPR-Cas Protein Sequence Classification

Nammi,

Madugula,

Pujar

et al. 2024

Preprint

View full text Add to dashboard Cite

The discovery of the CRISPR-Cas system has significantly advanced genome editing, offering vast applications in medical treatments and life sciences research. Despite their immense potential, the existing CRISPR-Cas proteins still face challenges concerning size, delivery efficiency, and cleavage specificity. Addressing these challenges necessitates a deeper understanding of CRISPR-Cas proteins to enhance the design and discovery of novel Cas proteins for precision gene editing. In this study, we performed extensive deep-learning research on CRISPR-Cas proteins, aiming to develop a classification model capable of distinguishing CAS from non-CAS proteins, as well as discriminating sub-categories of CAS proteins, specifically CAS9 and CAS12. We developed two types of deep learning models: 1) a transformer encoder-based classification model, trained from scratch; and 2) a large protein language model fine-tuned on ProtBert, pre-trained on more than 200 million proteins. To boost learning efficiency for the model trained from scratch, we introduced a novel margin-based loss function to maximize inter-class separability and intra-class compactness in protein sequence embedding latent space of a transformer encoder. The experimental results show that the Fine-Tuned ProtBert-based (FTPB) classification model achieved accuracies of 99.06\%, 94.42\%, 96.80\%, 97.57\% for CAS9 vs. Non-CAS, CAS12 vs. Non-CAS, CAS9 vs. CAS12, and multi-class classification of CAS9 vs. CAS12 vs. Non-CAS, respectively. The Latent Space Regularized Max-Margin Transformer (LSRMT) model achieved classification accuracies of 99.81\%, 99.81\%, 99.06\%, 99.27\% for the same tasks, respectively. These results demonstrate the effectiveness of the proposed Max-Margin-based latent space regularization in enhancing model robustness and generalization capabilities. Remarkably, the LSRMT model, even when trained on a significantly smaller dataset, outperformed the fine-tuned state-of-the-art large protein model. The high classification accuracies achieved by the LSRMT model demonstrate its proficiency in identifying discriminative features of CAS proteins, marking a significant step towards advancing our understanding of CAS protein structures in future research endeavors.

show abstract

Section: Related Workmentioning

confidence: 99%

Transformer-Based Deep Learning Model with Latent Space Regularization for CRISPR-Cas Protein Sequence Classification

Nammi,

Madugula,

Pujar

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Studies on transformer architecture [ 13 ] have demonstrated its efficacy in tackling large-scale computing challenges posed by excessively long sequences, surpassing CNNs in various tasks. For instance, Du et al proposed a DL model for predicting secretory proteins in plasma and saliva [ 14 ]. Shao et al learned complex features from protein sequence information through a CNN, a bidirectional gated recurrent unit (BGRU), and other networks, and completed the prediction of human body fluids.…”

Section: Introductionmentioning

confidence: 99%

ESMSec: Prediction of Secreted Proteins in Human Body Fluids Using Protein Language Models and Attention

Wang,

Sun,

Sheng

et al. 2024

IJMS

View full text Add to dashboard Cite

The secreted proteins of human body fluid have the potential to be used as biomarkers for diseases. These biomarkers can be used for early diagnosis and risk prediction of diseases, so the study of secreted proteins of human body fluid has great application value. In recent years, the deep-learning-based transformer language model has transferred from the field of natural language processing (NLP) to the field of proteomics, leading to the development of protein language models (PLMs) for protein sequence representation. Here, we propose a deep learning framework called ESM Predict Secreted Proteins (ESMSec) to predict three types of proteins secreted in human body fluid. The ESMSec is based on the ESM2 model and attention architecture. Specifically, the protein sequence data are firstly put into the ESM2 model to extract the feature information from the last hidden layer, and all the input proteins are encoded into a fixed 1000 × 480 matrix. Secondly, multi-head attention with a fully connected neural network is employed as the classifier to perform binary classification according to whether they are secreted into each body fluid. Our experiment utilized three human body fluids that are important and ubiquitous markers. Experimental results show that ESMSec achieved average accuracy of 0.8486, 0.8358, and 0.8325 on the testing datasets for plasma, cerebrospinal fluid (CSF), and seminal fluid, which on average outperform the state-of-the-art (SOTA) methods. The outstanding performance results of ESMSec demonstrate that the ESM can improve the prediction performance of the model and has great potential to screen the secretion information of human body fluid proteins.

show abstract

“…Despite these models achieving promising performances, they generally suffered from some limitations such as manual intervention in the feature selection procedures. Recently, deep learning (DL) with neural network models, such as convolutional neural network (CNN), long short-term memory (LSTM) and gated recurrent unit (GRU), have been used for body-fluid protein prediction [11][12][13]. Du et al proposed a DL model, named DeepUEP, which consists of a CNN module, a recurrent neural network (RNN) with LSTM and an attention module to predict the urine excretory proteins [12].…”

Section: Introductionmentioning

confidence: 99%

“…Du et al proposed a DL model, named DeepUEP, which consists of a CNN module, a recurrent neural network (RNN) with LSTM and an attention module to predict the urine excretory proteins [12]. In addition, they proposed a DL model based on the capsule network and Transformer architecture, SecProCT, to predict secretory proteins in blood and saliva [13]. Additionally, a novel DL framework, DeepSec, our previous research, for prediction of 12 different types of human body fluids was presented using CNN and a bidirectional gated recurrent unit (BGRU) [11].…”

Section: Introductionmentioning

confidence: 99%

DenSec: Secreted Protein Prediction in Cerebrospinal Fluid Based on DenseNet and Transformer

Huang

et al. 2022

Mathematics

View full text Add to dashboard Cite

Cerebrospinal fluid (CSF) exists in the surrounding spaces of mammalian central nervous systems (CNS); therefore, there are numerous potential protein biomarkers associated with CNS disease in CSF. Currently, approximately 4300 proteins have been identified in CSF by protein profiling. However, due to the diverse modifications, as well as the existing technical limits, large-scale protein identification in CSF is still considered a challenge. Inspired by computational methods, this paper proposes a deep learning framework, named DenSec, for secreted protein prediction in CSF. In the first phase of DenSec, all input proteins are encoded as a matrix with a fixed size of 1000 × 20 by calculating a position-specific score matrix (PSSM) of protein sequences. In the second phase, a dense convolutional network (DenseNet) is adopted to extract the feature from these PSSMs automatically. After that, Transformer with a fully connected dense layer acts as classifier to perform a binary classification in terms of secretion into CSF or not. According to the experiment results, DenSec achieves a mean accuracy of 86.00% in the test dataset and outperforms the state-of-the-art methods.

show abstract

SecProCT: In Silico Prediction of Human Secretory Proteins Based on Capsule Network and Transformer

Cited by 10 publications

References 47 publications

Transformer-Based Deep Learning Model with Latent Space Regularization for CRISPR-Cas Protein Sequence Classification

Transformer-Based Deep Learning Model with Latent Space Regularization for CRISPR-Cas Protein Sequence Classification

ESMSec: Prediction of Secreted Proteins in Human Body Fluids Using Protein Language Models and Attention

DenSec: Secreted Protein Prediction in Cerebrospinal Fluid Based on DenseNet and Transformer

Contact Info

Product

Resources

About