Transfer learning identifies sequence determinants of regulatory element accessibility

Salvatore, Marco; Horlacher, Marc; Winther, Ole; Andersson, Robin

doi:10.1101/2022.08.05.502903

Cited by 4 publications

(6 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CNN-based enhancer modeling has recently gained traction, due to their capacity for the interpretation of enhancer grammar 12, 19, 20, 74 A key limitation of CNN models is that they require large input data sets for training. While training on small data sets may lead to overfitting, transfer learning from sequence models trained with large data sets has been recently shown as a robust alternative 75 , Here, we propose several transfer learning applications, whereby the first (topic-based) model is fine-tuned either to learn cell state (in our case, hepatocyte zonation) or enhancer activity (based on MPRA data). The topic-CNN could recapitulate the core hepatocyte code, with sequence features associated with the same TFs as identified by SCENIC+.…”

Section: Discussionmentioning

confidence: 99%

Enhancer grammar of liver cell types and hepatocyte zonation states

González-Blas

Matetovici

Hillen

et al. 2022

Preprint

View full text Add to dashboard Cite

Cell type identity is encoded by gene regulatory networks (GRN), in which transcription factors (TFs) bind to enhancers to regulate target gene expression. In the mammalian liver, lineage TFs have been characterized for the main cell types, including hepatocytes. Hepatocytes cover a relatively broad cellular state space, as they differ significantly in their metabolic state, and function, depending on their position with respect to the central or portal vein in a liver lobule. It is unclear whether this spatially defined cellular state space, called zonation, is also governed by a well-defined gene regulatory code. To address this challenge, we have mapped enhancer-GRNs across liver cell types at high resolution, using a combination of single cell multiomics, spatial omics, GRN inference, and deep learning. We found that cell state changes in transcription and chromatin accessibility in hepatocytes, liver sinusoidal endothelial cells and hepatic stellate cells depend on zonation. Enhancer-GRN mapping suggests that zonation states in hepatocytes are driven by the repressors Tcf7l1 and Tbx3, that modulate the core hepatocyte GRN, controlled by Hnf4a, Cebpa, Hnf1a, Onecut1 and Foxa1, among others. To investigate how these TFs cooperate with cell type TFs, we performed an in vivo massively parallel reporter assay on 12,000 hepatocyte enhancers and used these data to train a hierarchical deep learning model (called DeepLiver) that exploits both enhancer accessibility and activity. DeepLiver confirms Cebpa, Onecut, Foxa1, Hnf1a and Hnf4a as drivers of enhancer specificity in hepatocytes; Tcf7l1/2 and Tbx3 as regulators of the zonation state; and Hnf4a, Hnf1a, AP-1 and Ets as activators. Finally, taking advantage of in silico mutagenesis predictions from DeepLiver and enhancer assays, we confirmed that the destruction of Tcf7l1/2 or Tbx3 motifs in zonated enhancers abrogates their zonation bias. Our study provides a multi-modal understanding of the regulatory code underlying hepatocyte identity and their zonation state, that can be exploited to engineer enhancers with specific activity levels and zonation patterns.

show abstract

Section: Discussionmentioning

confidence: 99%

Enhancer grammar of liver cell types and hepatocyte zonation states

González-Blas

Matetovici

Hillen

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For each of the 27 genomics tasks, we compared the performance of ChatNT with the state-of-the-art method for the respective dataset. These included the convolutional neural networks DeepSTARR [11], ChromTransfer [56], APARENT2 [21] and Saluki [19]; and the fine-tuned foundation models based on Nucleotide Transformer [26], agroNT [55], DNABERT [27] and ESM2 [48]. We used different performance metrics per task to follow the same metric used in the respective studies.…”

Section: Baselines For the Genomics Tasksmentioning

confidence: 99%

“…This collection includes a non-redundant subset of tasks from the Nucleotide Transformer [26] and the BEND [54] benchmarks, complemented with relevant tasks from the plant AgroNT benchmark [55] and human ChromTransfer [56]. These benchmarks have been extensively used in the literature, come from different research groups, and represent diverse DNA processes and species.…”

Section: A New Curated Genomics Instructions Dataset Of Biologically ...mentioning

confidence: 99%

See 1 more Smart Citation

ChatNT: A Multimodal Conversational Agent for DNA, RNA and Protein Tasks

Richard,

de Almeida,

Dalla-Torre

et al. 2024

Preprint

View full text Add to dashboard Cite

Language models are thriving, powering conversational agents that assist and empower humans to solve a number of tasks. Recently, these models were extended to support additional modalities including vision, audio and video, demonstrating impressive capabilities across multiple domains including healthcare. Still, conversational agents remain limited in biology as they cannot yet fully comprehend biological sequences. On the other hand, high-performance foundation models for biological sequences have been built through self-supervision over sequencing data, but these need to be fine-tuned for each specific application, preventing transfer and generalization between tasks. In addition, these models are not conversational which limits their utility to users with coding capabilities. In this paper, we propose to bridge the gap between biology foundation models and conversational agents by introducing ChatNT, the first multimodal conversational agent with an advanced understanding of biological sequences. ChatNT achieves new state-of-the-art results on the Nucleotide Transformer benchmark while being able to solve all tasks at once, in English, and to generalize to unseen questions. In addition, we have curated a new set of more biologically relevant instructions tasks from DNA, RNA and proteins, spanning multiple species, tissues and biological processes. ChatNT reaches performance on par with state-of-the-art specialized methods on those tasks. We also present a novel perplexity-based technique to help calibrate the confidence of our model predictions. Our framework for genomics instruction-tuning can be easily extended to more tasks and biological data modalities (e.g. structure, imaging), making it a widely applicable tool for biology. ChatNT is the first model of its kind and constitutes an initial step towards building generally capable agents that understand biology from first principles while being accessible to users with no coding background.

show abstract

“…While steady-state models can use all detected REs (~100s of thousands), only a subset of these REs are expected to respond to perturbation of a single TF (<10s of thousands), limiting the number of available training examples. Transfer learning, in which deep learning models are "pre-trained" on a larger set of related examples and then finetuned to predict the desired task, has recently emerged as an attractive solution to this type of problem, enabling use of deep learning models in data-limited settings [15][16][17][18][19] .…”

Section: Introductionmentioning

confidence: 99%

Transfer learning reveals sequence determinants of the quantitative response to transcription factor dosage

Naqvi,

Kim,

Tabatabaee

et al. 2024

Preprint

View full text Add to dashboard Cite

Deep learning approaches have made significant advances in predicting cell type-specific chromatin patterns from the identity and arrangement of transcription factor (TF) binding motifs. However, most models have been applied in unperturbed contexts, precluding a predictive understanding of how chromatin state responds to TF perturbation. Here, we used transfer learning to train and interpret deep learning models that use DNA sequence to predict, with accuracy approaching experimental reproducibility, how the concentration of two dosage-sensitive TFs (TWIST1, SOX9) affects regulatory element (RE) chromatin accessibility in facial progenitor cells. High-affinity motifs that allow for heterotypic TF co-binding and are concentrated at the center of REs buffer against quantitative changes in TF dosage and strongly predict unperturbed accessibility. In contrast, motifs with low-affinity or homotypic binding distributed throughout REs lead to sensitive responses with minimal contributions to unperturbed accessibility. Both buffering and sensitizing features show signatures of purifying selection. We validated these predictive sequence features using reporter assays and showed that a biophysical model of TF-nucleosome competition can explain the sensitizing effect of low-affinity motifs. Our approach of combining transfer learning and quantitative measurements of the chromatin response to TF dosage therefore represents a powerful method to reveal additional layers of the cis-regulatory code.

show abstract

Transfer learning identifies sequence determinants of regulatory element accessibility

Cited by 4 publications

References 59 publications

Enhancer grammar of liver cell types and hepatocyte zonation states

Enhancer grammar of liver cell types and hepatocyte zonation states

ChatNT: A Multimodal Conversational Agent for DNA, RNA and Protein Tasks

Transfer learning reveals sequence determinants of the quantitative response to transcription factor dosage

Contact Info

Product

Resources

About