Predicting gene and protein expression levels from DNA and protein sequences with Perceiver

Abstract

The functions of an organism and its biological processes derive from the expression and activity of genes and proteins. Therefore quantifying and predicting gene and protein expression values is a crucial aspect of scientific research. Concerning the prediction of gene expression values, the available machine learning-based approaches use the gene sequence %with the succession of nitrogenous bases as inputs to the neural network models. Some techniques, including Xpresso and Basenjii, have been proposed to predict gene expression values in the samples. However, these architectures are mainly based on Convolutional or Long Short Term Memory Networks, neglecting the attention mechanisms impact in selecting the sequence's relevant portions for prediction purposes. In addition, as far as we know, there is no model for predicting protein expression values exploiting the sequence of genes (sequence of nitrogenous bases) or proteins (sequence of amino acids). Here, we present a new Perceiver-type model, which exploits a transformer-based architecture to profit from the attention module and overcome the quadratic complexity of the standard attention-based architectures. The contributions of this work are the following: 1. Development of the DNAperceiver model for improving the prediction of gene expression starting from the genes' sequence; 2. Development of a ProteinPerceiver model for predicting protein expression values starting from the amino acid sequence; 3. Development of a Protein\&DNAPerceiver model for predicting protein expression values by simultaneously combining the sequence of nitrogenous bases and amino acids. 4. Evaluation of models under multiple conditions: predicting values on cell lines, mice, and tumor tissues of glioblastoma and lung cancer. The results show the effectiveness of the Perceiver-type models in predicting gene and protein expression values starting from the gene/protein sequence. In this context, inserting further regulatory and epigenetic information into the model could improve the prediction task.

E2VD: a unified evolution-driven framework for virus variation drivers prediction

“…Taking all lineages together, CNN’s OPP is improved by 14.3% by E2VD. The situation is similar in expression prediction task (Fig.5c), where E2VD outperforms the other two competitors on most lineages, with the OPP achieving an 10.2% improvement over DNAPerceiver [25] for the mixture of all lineages. Over-all, E2VD surpasses state-of-the-art methods across the board on out-of-distribution lineages, demonstrating enhanced generalization performance.…”

“…The accuracy of the state-of-the-art method NN MM-GBSA [20] in the classification task is improved by 10.7% by E2VD, along with a 8% improvement of the Pearson’s Correlation Coefficient (PCC) in the regression task. Second, for expression prediction, three models from two previous studies [25, 26] are used for comparison. The accuracy of the state-of-the-art method DNAPerceiver [25] in the classification task is improved by E2VD by 20.8%, and its PCC in the regression task is improved by 13.5%.…”

“…Second, for expression prediction, three models from two previous studies [25, 26] are used for comparison. The accuracy of the state-of-the-art method DNAPerceiver [25] in the classification task is improved by E2VD by 20.8%, and its PCC in the regression task is improved by 13.5%. Finally, for antibody escape prediction, six models from two previous studies [23, 27] are used for comparison.…”

“…Secondly, we analyze the methods under the classification system of the implemented functions. Current specific property prediction methods [19, 20, 22, 23, 25–27] lack comprehensiveness and unification, and often only develop one or two customized predictors to achieve prediction of specific properties. Such methods are not suitable for studying virus evolution because they do not comprehensively consider diverse variation drivers.…”

Leveraging transformers‐based language models in proteome bioinformatics

“…Recent studies have used transformer‐based models to analyze protein expression data, particularly in drug discovery and development. For instance, Stefatini utilized transformers architecture to predict gene and protein expression levels from DNA and protein sequences [56]. At the same time, DeepMind proposed a model called Enformer to study how non‐coding DNA influences gene expression in different cell types [84].…”