Application of Sequence Embedding in Protein Sequence-Based Predictions

Ibtehaz, Nabil

doi:10.1142/9789811258589_0002

Cited by 5 publications

(6 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zscale, MS-WHIM, Stscale [ 10 , 34 ]) rather than specific features for each residue in the sequence, as was done for rs3DDPDs given the heavy influence of the environment in the dynamic behavior of single residues. On the other hand, protein embeddings are often the byproduct of a machine or deep learning model using a protein sequence as input [ 12 , 35 ], unlike the approach followed for ps3DDPDs. Here, instead, a common main framework was kept to increase the interpretability and interoperability of the resulting descriptors.…”

Section: Discussionmentioning

confidence: 99%

“…Descriptors derived from the protein sequence include discrete features calculated per residue (one-hot encoding) [ 10 ] or protein [ 11 ] capturing physicochemical properties or amino acid composition. Additionally, deep learning applications of natural language processing have prompted the generation of protein embeddings from sequences [ 12 ]. Structure-based descriptors can be derived from molecular graphs or the protein 3D structure by measuring connectivity, distances, and physicochemical properties among others [ 8 , 9 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

3DDPDs: describing protein dynamics for proteochemometric bioactivity prediction. A case for (mutant) G protein-coupled receptors

Gorostiola González,

van den Broek,

Braun

et al. 2023

J Cheminform

View full text Add to dashboard Cite

Proteochemometric (PCM) modelling is a powerful computational drug discovery tool used in bioactivity prediction of potential drug candidates relying on both chemical and protein information. In PCM features are computed to describe small molecules and proteins, which directly impact the quality of the predictive models. State-of-the-art protein descriptors, however, are calculated from the protein sequence and neglect the dynamic nature of proteins. This dynamic nature can be computationally simulated with molecular dynamics (MD). Here, novel 3D dynamic protein descriptors (3DDPDs) were designed to be applied in bioactivity prediction tasks with PCM models. As a test case, publicly available G protein-coupled receptor (GPCR) MD data from GPCRmd was used. GPCRs are membrane-bound proteins, which are activated by hormones and neurotransmitters, and constitute an important target family for drug discovery. GPCRs exist in different conformational states that allow the transmission of diverse signals and that can be modified by ligand interactions, among other factors. To translate the MD-encoded protein dynamics two types of 3DDPDs were considered: one-hot encoded residue-specific (rs) and embedding-like protein-specific (ps) 3DDPDs. The descriptors were developed by calculating distributions of trajectory coordinates and partial charges, applying dimensionality reduction, and subsequently condensing them into vectors per residue or protein, respectively. 3DDPDs were benchmarked on several PCM tasks against state-of-the-art non-dynamic protein descriptors. Our rs- and ps3DDPDs outperformed non-dynamic descriptors in regression tasks using a temporal split and showed comparable performance with a random split and in all classification tasks. Combinations of non-dynamic descriptors with 3DDPDs did not result in increased performance. Finally, the power of 3DDPDs to capture dynamic fluctuations in mutant GPCRs was explored. The results presented here show the potential of including protein dynamic information on machine learning tasks, specifically bioactivity prediction, and open opportunities for applications in drug discovery, including oncology.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

3DDPDs: describing protein dynamics for proteochemometric bioactivity prediction. A case for (mutant) G protein-coupled receptors

Gorostiola González,

van den Broek,

Braun

et al. 2023

J Cheminform

View full text Add to dashboard Cite

show abstract

“…We generate the sequence “[BOS], M 1 , …, M p , [SEP], P 1 , …, P q , [EOS]” with length p + q + 3 as the ESM model input, and obtain the same size embedding vectors

from the last layer of ESM models, corresponding to the special tokens and the amino acids in the MHC and the peptide. As a common strategy in NLP sequence classification tasks, we use the embedding of [BOS] to be the MHC-peptide sequence-pair embedding vector

( Ibtehaz and Kihara, 2023 ). Finally, passing

through a softmax classifier layer, we output the probability of binding and use it to compute the loss and apply back-propagation.…”

Section: Methodsmentioning

confidence: 99%

Improved prediction of MHC-peptide binding using protein language models

Hashemi,

Hao,

Ignatov

et al. 2023

Front. Bioinform.

View full text Add to dashboard Cite

Major histocompatibility complex Class I (MHC-I) molecules bind to peptides derived from intracellular antigens and present them on the surface of cells, allowing the immune system (T cells) to detect them. Elucidating the process of this presentation is essential for regulation and potential manipulation of the cellular immune system. Predicting whether a given peptide binds to an MHC molecule is an important step in the above process and has motivated the introduction of many computational approaches to address this problem. NetMHCPan, a pan-specific model for predicting binding of peptides to any MHC molecule, is one of the most widely used methods which focuses on solving this binary classification problem using shallow neural networks. The recent successful results of Deep Learning (DL) methods, especially Natural Language Processing (NLP-based) pretrained models in various applications, including protein structure determination, motivated us to explore their use in this problem. Specifically, we consider the application of deep learning models pretrained on large datasets of protein sequences to predict MHC Class I-peptide binding. Using the standard performance metrics in this area, and the same training and test sets, we show that our models outperform NetMHCpan4.1, currently considered as the-state-of-the-art.

show abstract

“…The concept of embeddings originally stems from natural language processing (NLP) and language models, but targeted adaptations for amino acid sequence data have already been presented [Ibtehaz and Kihara, 2021]. By associating numerical vectors to tokens such as characters or words, embeddings constitute the first encoding step of a model.…”

Section: Benchmarking Of Proteasomal Cleavage Prediction Strategiesmentioning

confidence: 99%

“…By associating numerical vectors to tokens such as characters or words, embeddings constitute the first encoding step of a model. The method employed to generate embeddings is crucial as it influences what intrinsic information a model can exploit for classification [Ibtehaz and Kihara, 2021]. We, thus, considered various embeddings in our analysis, while keeping the base architecture equal.…”

Section: Benchmarking Of Proteasomal Cleavage Prediction Strategiesmentioning

confidence: 99%

Proteasomal cleavage prediction: state-of-the-art and future directions

Ziegler

Bolei

Bischl

et al. 2023

Preprint

View full text Add to dashboard Cite

Epitope vaccines are a promising approach for precision treatment of pathogens, cancer, autoimmune diseases, and allergies. Effectively designing such vaccines requires accurate proteasomal cleavage prediction to ensure that the epitopes included in the vaccine trigger an immune response. The performance of proteasomal cleavage predictors has been steadily improving over the past decades owing to increasing data availability and methodological advances. In this review, we summarize the current proteasomal cleavage prediction landscape and, in light of recent progress in the field of deep learning, develop and compare a wide range of recent architectures and techniques, including long short-term memory (LSTM), transformers, and convolutional neural networks (CNN), as well as four different denoising techniques. All open-source cleavage predictors re-trained on our dataset performed within two AUC percentage points. Our comprehensive deep learning architecture benchmark improved performance by 0.4 AUC percentage points, while closed-source predictors performed considerably worse. We found that a wide range of architectures and training regimes all result in very similar performance, suggesting that the specific modeling approach employed has a limited impact on predictive performance compared to the specifics of the dataset employed. We speculate that the noise and implicit nature of data acquisition techniques used for training proteasomal cleavage prediction models and the complexity of biological processes of the antigen processing pathway are the major limiting factors. While biological complexity can be tackled by more data and, to a lesser extent, better models, noise and randomness inherently limit the maximum achievable predictive performance.

show abstract

Application of Sequence Embedding in Protein Sequence-Based Predictions

Cited by 5 publications

References 61 publications

3DDPDs: describing protein dynamics for proteochemometric bioactivity prediction. A case for (mutant) G protein-coupled receptors

3DDPDs: describing protein dynamics for proteochemometric bioactivity prediction. A case for (mutant) G protein-coupled receptors

Improved prediction of MHC-peptide binding using protein language models

Proteasomal cleavage prediction: state-of-the-art and future directions

Contact Info

Product

Resources

About