Evolutionary Context-Integrated Deep Sequence Modeling for Protein Engineering

Luo, Ying; Vo, Lam; Ding, Hantian; Su, Yufeng; Liu, Yang; Qian, Wesley Wei; Zhao, Huimin; Peng, Jian

doi:10.1007/978-3-030-45257-5_30

Cited by 8 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The intermediate contact map representation in the model is directly interpretable and can be used to validate the prediction or study the proteins' binding regions on a residue scale. D-SCRIPT thus joins the small but growing set of advances in interpretable deep-learning methods in computational biology (Hie et al, 2020;Luo et al, 2020aLuo et al, , 2020b. Our modular design additionally enables the investigation of model output at various stages, and we demonstrate that each layer captures incremental structural information.…”

Section: Discussionmentioning

confidence: 89%

See 1 more Smart Citation

D-SCRIPT translates genome to phenome with sequence-based, structure-aware, genome-scale predictions of protein-protein interactions

et al. 2021

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 89%

“…Chen et al's in PIPR), where each amino acid's embedding represents just its biochemical properties or a short-range context (e.g., 5-7 residues) around it. We note that alternative embeddings (Rives et al, 2019;Luo et al, 2020aLuo et al, , 2020b can potentially be substituted here.…”

Section: Model Architecturementioning

confidence: 99%

D-SCRIPT translates genome to phenome with sequence-based, structure-aware, genome-scale predictions of protein-protein interactions

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Protein engineering with UniRep ( 30 ) showed that general global protein representations can support training function-specific supervised models with relatively few sequence–function examples. ECNet pioneered an approach for combining global protein representations, local information about residue coevolution, and protein sequence features ( 31 ). Across tens of deep mutational scanning datasets, ECNet was almost always superior to unsupervised learning models and models based only on a global protein representation.…”

Section: Discussionmentioning

confidence: 99%

Neural networks to learn protein sequence–function relationships from deep mutational scanning data

Gelman

Fahlberg

Heinzelman

et al. 2021

Proc. Natl. Acad. Sci. U.S.A.

110

146

View full text Add to dashboard Cite

The mapping from protein sequence to function is highly complex, making it challenging to predict how sequence changes will affect a protein’s behavior and properties. We present a supervised deep learning framework to learn the sequence–function mapping from deep mutational scanning data and make predictions for new, uncharacterized sequence variants. We test multiple neural network architectures, including a graph convolutional network that incorporates protein structure, to explore how a network’s internal representation affects its ability to learn the sequence–function mapping. Our supervised learning approach displays superior performance over physics-based and unsupervised prediction methods. We find that networks that capture nonlinear interactions and share parameters across sequence positions are important for learning the relationship between sequence and function. Further analysis of the trained models reveals the networks’ ability to learn biologically meaningful information about protein structure and mechanism. Finally, we demonstrate the models’ ability to navigate sequence space and design new proteins beyond the training set. We applied the protein G B1 domain (GB1) models to design a sequence that binds to immunoglobulin G with substantially higher affinity than wild-type GB1.

show abstract

“…Bepler & Berger (2019) pre-trained LSTMs on protein sequences, adding supervision from contacts to produce embeddings. Subsequent to our preprint, related works have built on its exploration of protein sequence modeling, exploring generative models (Riesselman et al, 2019;Madani et al, 2020), internal representations of Transformers (Vig et al, 2020), and applications of representation learning and generative modeling such as classification (Elnaggar et al, 2019;Strodthoff et al, 2020), mutational effect prediction (Luo et al, 2020), and design of sequences (Repecka et al, 2019;Hawkins-Hooker et al, 2020;Amimeur et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences

Rives

Goyal

Meier

et al. 2019

Preprint

496

986

View full text Add to dashboard Cite

In the field of artificial intelligence, a combination of scale in data and model capacity enabled by unsupervised learning has led to major advances in representation learning and statistical generation. In biology, the anticipated growth of sequencing promises unprecedented data on natural sequence diversity. Learning the natural distribution of evolutionary protein sequence variation is a logical step toward predictive and generative modeling for biology. To this end we use unsupervised learning to train a deep contextual language model on 86 billion amino acids across 250 million sequences spanning evolutionary diversity. The resulting model maps raw sequences to representations of biological properties without labels or prior domain knowledge. The learned representation space organizes sequences at multiple levels of biological granularity from the biochemical to proteomic levels. Unsupervised learning recovers information about protein structure: secondary structure and residue-residue contacts can be identified by linear projections from the learned representations. Training language models on full sequence diversity rather than individual protein families increases recoverable information about secondary structure. The unsupervised models can be adapted with supervision from quantitative mutagenesis data to predict variant activity. Predictions from sequences alone are comparable to results from a state-of-the-art model of mutational effects that uses evolutionary and structurally derived features.

show abstract

Evolutionary Context-Integrated Deep Sequence Modeling for Protein Engineering

Cited by 8 publications

References 0 publications

D-SCRIPT translates genome to phenome with sequence-based, structure-aware, genome-scale predictions of protein-protein interactions

D-SCRIPT translates genome to phenome with sequence-based, structure-aware, genome-scale predictions of protein-protein interactions

Neural networks to learn protein sequence–function relationships from deep mutational scanning data

Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences

Contact Info

Product

Resources

About