Sequence modeling and design from molecular to genome scale with Evo

Nguyen, Eric; Poli, Michael; Durrant, Matthew G; Thomas, Armin W; Kang, Brian; Sullivan, Jeremy; Ng, Madelena Y; Lewis, Ashley; Patel, Aman; Lou, Aaron; Ermon, Stefano; Baccus, Stephen A; Hernandez-Boussard, Tina; Re, Christopher; Hsu, Patrick D; Hie, Brian L

doi:10.1101/2024.02.27.582234

Cited by 43 publications

(12 citation statements)

References 115 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It does, however, suggest that a path forward for the generation of whole genomes that resemble members of existing virus families may require the development of family-specific models. This may also be achieved by prepending taxonomic labels to sequences in the training dataset, as done elsewhere (Nguyen et al, 2024). However, the low data availability for the vast majority of viral families will challenge their development in the short-term.…”

Section: Discussionmentioning

confidence: 99%

Transformer Model Generated Bacteriophage Genomes are Compositionally Distinct from Natural Sequences

Ratcliff

2024

Preprint

View full text Add to dashboard Cite

Novel applications of language models in genomics promise to have a large impact on the field. The megaDNA model is the first publicly available generative model for creating synthetic viral genomes. To evaluate megaDNA’s ability to recapitulate the nonrandom genome composition of viruses and assess whether synthetic genomes can be algorithmically detected, compositional metrics for 4,969 natural bacteriophage genomes and 1,002de novosynthetic bacteriophage genomes were compared. Transformer-generated sequences had varied but realistic genome lengths and 58% were classified as viral by geNomad. However, the sequences demonstrated consistent differences in various compositional metrics when compared to natural bacteriophage genomes by rank-sum tests and principal component analysis. A simple neural network trained to detect transformer-generated sequences on global compositional metrics alone displayed a median sensitivity of 93.0% and specificity of 97.9% (n = 12 independent models). Overall, these results demonstrate that megaDNA does not yet generate bacteriophage genomes with realistic compositional biases and that genome composition is a reliable method for detecting sequences generated by this model. While the results are specific to the megaDNA model, the evaluate framework described here could be applied to any generative model for genomic sequences.

show abstract

Section: Discussionmentioning

confidence: 99%

Transformer Model Generated Bacteriophage Genomes are Compositionally Distinct from Natural Sequences

Ratcliff

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…While this work has focused on adaptation of PLMs, large protein structure models such as OmegaFold, RosettaFold, ESMFold, and AlphaFold ( 3 , 27 – 29 ) have begun to see use as foundation models and could be similarly amenable to tuning for downstream tasks with PEFT methods. In addition, it remains to be shown whether PEFT methods are equally competitive for language models trained on other types of biological sequences, such as DNA ( 46 ) or SMILES strings ( 22 ).…”

Section: Discussionmentioning

confidence: 99%

Democratizing protein language models with parameter-efficient fine-tuning

Sledzieski,

Kshirsagar,

Baek

et al. 2024

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Proteomics has been revolutionized by large protein language models (PLMs), which learn unsupervised representations from large corpora of sequences. These models are typically fine-tuned in a supervised setting to adapt the model to specific downstream tasks. However, the computational and memory footprint of fine-tuning (FT) large PLMs presents a barrier for many research groups with limited computational resources. Natural language processing has seen a similar explosion in the size of models, where these challenges have been addressed by methods for parameter-efficient fine-tuning (PEFT). In this work, we introduce this paradigm to proteomics through leveraging the parameter-efficient method LoRA and training new models for two important tasks: predicting protein–protein interactions (PPIs) and predicting the symmetry of homooligomer quaternary structures. We show that these approaches are competitive with traditional FT while requiring reduced memory and substantially fewer parameters. We additionally show that for the PPI prediction task, training only the classification head also remains competitive with full FT, using five orders of magnitude fewer parameters, and that each of these methods outperform state-of-the-art PPI prediction methods with substantially reduced compute. We further perform a comprehensive evaluation of the hyperparameter space, demonstrate that PEFT of PLMs is robust to variations in these hyperparameters, and elucidate where best practices for PEFT in proteomics differ from those in natural language processing. All our model adaptation and evaluation code is available open-source at https://github.com/microsoft/peft_proteomics . Thus, we provide a blueprint to democratize the power of PLM adaptation to groups with limited computational resources.

show abstract

“…reduce changes in promoter activity, regulatory functions, and translation rate and avoid generating unwanted cryptic promoters and translated ORFs. The development of predictive models capable of forecasting transcriptional and translational effects and integrating these predictions to refactor codon composition is expected to minimize the negative fitness impact of synonymous genome recoding [83][84][85][86][87][88][89][90][91][92] .…”

Section: Future Genome Design Projects Should Focus On Developing And...mentioning

confidence: 99%

Synthetic genomes unveil the effects of synonymous recoding

Nyerges,

Chiappino-Pepe,

Budnik

et al. 2024

Preprint

View full text Add to dashboard Cite

Engineering the genetic code of an organism provides the basis for (i) making any organism safely resistant to natural viruses and (ii) preventing genetic information flow into and out of genetically modified organisms while (iii) allowing the biosynthesis of genetically encoded unnatural polymers1–4. Achieving these three goals requires the reassignment of multiple of the 64 codons nature uses to encode proteins. However, synonymous codon replacement—recoding—is frequently lethal, and how recoding impacts fitness remains poorly explored. Here, we explore these effects using whole-genome synthesis, multiplexed directed evolution, and genome-transcriptome-translatome-proteome co-profiling on multiple recoded genomes. Using this information, we assemble a syntheticEscherichia coligenome in seven sections using only 57 codons to encode proteins. By discovering the rules responsible for the lethality of synonymous recoding and developing a data-driven multi-omics-based genome construction workflow that troubleshoots synthetic genomes, we overcome the lethal effects of 62,007 synonymous codon swaps and 11,108 additional genomic edits. We show that synonymous recoding induces transcriptional noise including new antisense RNAs, leading to drastic transcriptome and proteome perturbation. As the elimination of select codons from an organism’s genetic code results in the widespread appearance of cryptic promoters, we show that synonymous codon choice may naturally evolve to minimize transcriptional noise. Our work provides the first genome-scale description of how synonymous codon changes influence organismal fitness and paves the way for the construction of functional genomes that provide genetic firewalls from natural ecosystems and safely produce biopolymers, drugs, and enzymes with an expanded chemistry.

show abstract

Sequence modeling and design from molecular to genome scale with Evo

Cited by 43 publications

References 115 publications

Transformer Model Generated Bacteriophage Genomes are Compositionally Distinct from Natural Sequences

Transformer Model Generated Bacteriophage Genomes are Compositionally Distinct from Natural Sequences

Democratizing protein language models with parameter-efficient fine-tuning

Synthetic genomes unveil the effects of synonymous recoding

Contact Info

Product

Resources

About