Predicting gene expression levels from DNA sequences and post-transcriptional information with transformers

Pipoli, Vittorio; Cappelli, Mattia; Palladini, A.; Peluso, Carlo; Lovino, Marta; Ficarra, Elisa

doi:10.1016/j.cmpb.2022.107035

Cited by 9 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We tested four sequenced-based genomic deep learning architectures, DanQ (Quang & Xie, 2016), Hye-naDNA (Nguyen et al, 2023), FNetCompression (Pipoli et al, 2023), and a smaller version of Enformer (Žiga Avsec, Agarwal, et al, 2021), on their ability to predict across species and alleles. DanQ is one of the earliest genomic deep learning architectures, leveraging a long short-term memory recurrent layer to learn the syntax and grammar of motifs detected by a convolutional layer.…”

Section: Introductionmentioning

confidence: 99%

Current genomic deep learning architectures generalize across grass species but not alleles

Wrightsman,

Ferebee,

Romay

et al. 2024

Preprint

View full text Add to dashboard Cite

Non-coding regions of the genome are just as important as coding regions for understanding the mapping from genotype to phenotype. Interpreting deep learning models trained on RNA-seq is an emerging method to highlight functional sites within non-coding regions. Most of the work on RNA abundance models has been done within humans and mice, with little attention paid to plants. Here, we benchmark four genomic deep learning model architectures with genomes and RNA-seq data from 18 species closely related to maize and sorghum within the Andropogoneae. The Andropogoneae are a tribe of C4 grasses that have adapted to a wide range of environments worldwide since diverging 18 million years ago. Hundreds of millions of years of evolution across these species has produced a large, diverse pool of training alleles across species sharing a common physiology. As model input, we extracted 1,026 base pairs upstream of each gene’s translation start site. We held out maize as our test set and two closely related species as our validation set, training each architecture on the remaining Andropogoneae genomes. Within a panel of 26 maize lines, all architectures predict expression across genes moderately well but poorly across alleles. DanQ consistently ranked highest or second highest among all architectures yet performance was generally very similar across architectures despite orders of magnitude differences in size. This suggests that state-of-the-art supervised genomic deep learning models are able to generalize moderately well across related species but not sensitively separate alleles within species, the latter of which agrees with recent work within humans. We are releasing the preprocessed data and code for this work as a community benchmark to evaluate new architectures on our across-species and across-allele tasks.

show abstract

Section: Introductionmentioning

confidence: 99%

Current genomic deep learning architectures generalize across grass species but not alleles

Wrightsman,

Ferebee,

Romay

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…In recent years, deep learning techniques spread in health applications [9,10,11,12] [13,14] and previous works focused on mRNA level prediction from TSS-straddling sequences [15,16,17]. In particular, Convolutional Neural Networks have been adopted to deal with the sequential nature of the DNA [15,16,17,18,19]. Specifically, Basenjii [15] applies convolutional layers followed by dilated convolutions to share information across large distances in the gene sequences.…”

Section: Introductionmentioning

confidence: 99%

“…In this context, previous works focused on gene expression level prediction from nucleotide sequences exploiting deep learning techniques. In particular, various types of Convolutional Neural Networks have been adopted to deal with the sequential nature of the DNA [1,3,15,18,26]. Specifically, Basenjii [15] applies convolutional layers followed by dilated convolutions to share information across large distances.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Predicting gene and protein expression levels from DNA and protein sequences with Perceiver

Stefanini

Lovino

Cucchiara

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The functions of an organism and its biological processes derive from the expression and activity of genes and proteins. Therefore quantifying and predicting gene and protein expression values is a crucial aspect of scientific research. Concerning the prediction of gene expression values, the available machine learning-based approaches use the gene sequence %with the succession of nitrogenous bases as inputs to the neural network models. Some techniques, including Xpresso and Basenjii, have been proposed to predict gene expression values in the samples. However, these architectures are mainly based on Convolutional or Long Short Term Memory Networks, neglecting the attention mechanisms impact in selecting the sequence's relevant portions for prediction purposes. In addition, as far as we know, there is no model for predicting protein expression values exploiting the sequence of genes (sequence of nitrogenous bases) or proteins (sequence of amino acids). Here, we present a new Perceiver-type model, which exploits a transformer-based architecture to profit from the attention module and overcome the quadratic complexity of the standard attention-based architectures. The contributions of this work are the following: 1. Development of the DNAperceiver model for improving the prediction of gene expression starting from the genes' sequence; 2. Development of a ProteinPerceiver model for predicting protein expression values starting from the amino acid sequence; 3. Development of a Protein\&DNAPerceiver model for predicting protein expression values by simultaneously combining the sequence of nitrogenous bases and amino acids. 4. Evaluation of models under multiple conditions: predicting values on cell lines, mice, and tumor tissues of glioblastoma and lung cancer. The results show the effectiveness of the Perceiver-type models in predicting gene and protein expression values starting from the gene/protein sequence. In this context, inserting further regulatory and epigenetic information into the model could improve the prediction task.

show abstract

SynthCap: Augmenting Transformers with Synthetic Data for Image Captioning

Caffagni,

Barraco,

Cornia

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

Predicting gene expression levels from DNA sequences and post-transcriptional information with transformers

Cited by 9 publications

References 24 publications

Current genomic deep learning architectures generalize across grass species but not alleles

Current genomic deep learning architectures generalize across grass species but not alleles

Predicting gene and protein expression levels from DNA and protein sequences with Perceiver

SynthCap: Augmenting Transformers with Synthetic Data for Image Captioning

Contact Info

Product

Resources

About