UniDL4BioPep: a universal deep learning architecture for binary classification in peptide bioactivity

Abstract

Identification of potent peptides through model prediction can reduce benchwork in wet experiments. However, the conventional process of model buildings can be complex and time consuming due to challenges such as peptide representation, feature selection, model selection and hyperparameter tuning. Recently, advanced pretrained deep learning-based language models (LMs) have been released for protein sequence embedding and applied to structure and function prediction. Based on these developments, we have developed UniDL4BioPep, a universal deep-learning model architecture for transfer learning in bioactive peptide binary classification modeling. It can directly assist users in training a high-performance deep-learning model with a fixed architecture and achieve cutting-edge performance to meet the demands in efficiently novel bioactive peptide discovery. To the best of our best knowledge, this is the first time that a pretrained biological language model is utilized for peptide embeddings and successfully predicts peptide bioactivities through large-scale evaluations of those peptide embeddings. The model was also validated through uniform manifold approximation and projection analysis. By combining the LM with a convolutional neural network, UniDL4BioPep achieved greater performances than the respective state-of-the-art models for 15 out of 20 different bioactivity dataset prediction tasks. The accuracy, Mathews correlation coefficient and area under the curve were 0.7–7, 1.23–26.7 and 0.3–25.6% higher, respectively. A user-friendly web server of UniDL4BioPep for the tested bioactivities is established and freely accessible at https://nepc2pvmzy.us-east-1.awsapprunner.com. The source codes, datasets and templates of UniDL4BioPep for other bioactivity fitting and prediction tasks are available at https://github.com/dzjxzyd/UniDL4BioPep.

pLM4Alg: Protein Language Model-Based Predictors for Allergenic Proteins and Peptides

“…Another representative technique is the machine learning (ML) approach where the classification criterion is determined by the ML model by learning from a separate training data set. ,,,− , Besides the selection of modeling method, the most challenging task in the ML approach is the representation or encoding of protein/peptide sequences into a numerical vector/matrix. Various encoding methods, including amino acid descriptors, amino acid composition (AAC), pseudoamino acid composition (PseAAC), dipeptide composition (DPC), amino acid descriptors (AAD), position-specific scoring matrix (PSSM), physicochemical descriptors, biomedical properties, k-mer dictionary-based binary representation, etc., have been widely used in predicting allergenicity and other properties/bioactivities. ,,,,,− However, these features may not always accurately represent protein sequences and simple combinations can cause high-dimensional problems as well as the feature redundancy …”

“…Recently, inspired by the extraordinary performance of transformer-based models in natural language processing (NLP) tasks, a series of attempts have been made in pretrained protein language models (pLMs) (e.g., ProtTrans, ESM, UniRep, etc.) for downstream protein sequence tasks. ,− The central concept lies in incorporating the attention mechanism during the self-supervised learning process in these pLMs. This mechanism enables the model to observe the entire sequence simultaneously to learn the representation of each residue as well as their relationship, rather than locally (as in convolutional neural network (CNN)) or sequentially (as in long short-term networks (LSTM)).…”

“…Evolutionary scale modeling (ESM), one of the pretrained pLMs, was updated into ESM-2 in 2022 by the Meta Fundamental AI Research Protein Team (FAIR) for protein representation . In our previous study, it exhibited a great potential in developing a universal model architecture and achieved state-of-the-art (SOTA) performance in 15 out of 20 peptide bioactivity/function data sets by fitting the same model architecture in different peptide data sets . Due to the limited data availability of bioactive peptides and the universal application scenarios, a small size ESM-2 pLM (320 output dimension) and a simplified model architecture were selected .…”

Classification of bioactive peptides: a comparative analysis of models and encodings

“…Given the importance of BPs, there have been several attempts to create in-silico approaches to perform a preliminary assignment of the potential functional properties and facilitate the subsequent discovery and testing process in vivo [19–24]. These methods rely on several databases where peptides from various experiments have been collected and classified according to the BPs functional classes.…”

“…Tools include similarity-based classification using available sequences from databases [7,25] and prediction of physicochemical properties [26]. There have also been several attempts to use machine learning techniques to aid in the detection of BPs and their functional classification [19–24]. The proposed methods used Logistic Regression [27], Support Vector Machines [28] and Random Forests [29] to predict the BPs functional role.…”