ProtTrans-Glutar: Incorporating Features From Pre-trained Transformer-Based Models for Predicting Glutarylation Sites

Abstract

Lysine glutarylation is a post-translational modification (PTM) that plays a regulatory role in various physiological and biological processes. Identifying glutarylated peptides using proteomic techniques is expensive and time-consuming. Therefore, developing computational models and predictors can prove useful for rapid identification of glutarylation. In this study, we propose a model called ProtTrans-Glutar to classify a protein sequence into positive or negative glutarylation site by combining traditional sequence-based features with features derived from a pre-trained transformer-based protein model. The features of the model were constructed by combining several feature sets, namely the distribution feature (from composition/transition/distribution encoding), enhanced amino acid composition (EAAC), and features derived from the ProtT5-XL-UniRef50 model. Combined with random under-sampling and XGBoost classification method, our model obtained recall, specificity, and AUC scores of 0.7864, 0.6286, and 0.7075 respectively on an independent test set. The recall and AUC scores were notably higher than those of the previous glutarylation prediction models using the same dataset. This high recall score suggests that our method has the potential to identify new glutarylation sites and facilitate further research on the glutarylation process.

Codon language embeddings provide strong signals for protein engineering

“…Self-supervised training endows the latent variables of the model with highly informative features, known as learned representations, which can then be leveraged in downstream tasks where limited training data is available. Learned protein representations are currently central to the state-of-the-art tools for predicting variant fitness [3][4][5][6], protein function [7,8], subcellular localisation [9], solubility [10], binding sites [11], signal peptides [12], post-translational modifications [13], intrinsic disorder [14], and others [15,16], and they have shown promise in the path towards accurate alignment-free protein structure prediction [17][18][19][20][21]. Improving learned representations is therefore a potential path to deliver consistent, substantial improvements across computational protein engineering.…”

GBDT_KgluSite: An improved computational prediction model for lysine glutarylation sites based on Feature Fusion and GBDT classifier

“…For instance, iGluK-Deep [14] was proposed based on deep neural networks and Chou's Pseudo Amino Acid Composition (PseAAC). ProtTrans-Glutar [15] incorporated the XGBoost and pre-trained features using the transformer. Details of these studies are summarized in Table 1.…”

GBDT_KgluSite: An improved computational prediction model for lysine glutarylation sites based on feature fusion and GBDT classifier

“…For instance, iGluK-Deep [ 15 ] was proposed based on deep neural networks and Chou’s Pseudo Amino Acid Composition (PseAAC). ProtTrans-Glutar [ 16 ] incorporated the XGBoost and pre-trained features by Transformer. DeepDN_iGlu [ 17 ] was proposed by employing binary encoding as feature representation, using DenseNet as the classification model, and utilizing the focal loss function to address the imbalance issue.…”