The generative capacity of probabilistic protein sequence models

McGee, Francisco; Hauri, Sandro; Novinger, Quentin; Vučetić, Slobodan; Levy, Ronald M.; Carnevale, Vincenzo; Haldane, Allan

doi:10.1038/s41467-021-26529-9

Cited by 30 publications

(42 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We note that, without language models, analyzing the correlations in MSAs can reveal evolutionary relatedness and sub-families [15], as well as collective modes of correlation, some of which are phylogenetic and some functional [18]. Furthermore, Potts models capture the clustered organization of protein families in sequence space [26], and the latent space of variational autoencoder models trained on sequences [38][39][40] qualitatively captures phylogeny [39]. Here, we demonstrated the stronger result that detailed pairwise phylogenetic relationships between sequences are quantitatively learnt by MSA Transformer.…”

Section: Discussionmentioning

confidence: 99%

Protein language models trained on multiple sequence alignments learn phylogenetic relationships

Lupo

Sgarbossa

Bitbol

2022

Preprint

View full text Add to dashboard Cite

Self-supervised neural language models with attention have recently been applied to biological sequence data, advancing structure, function and mutational effect prediction. Some protein language models, including MSA Transformer and AlphaFold's EvoFormer, take multiple sequence alignments (MSAs) of evolutionarily related proteins as inputs. Simple combinations of MSA Transformer's row attentions have led to state-of-the-art unsupervised structural contact prediction. We demonstrate that similarly simple, and universal, combinations of MSA Transformer's column attentions strongly correlate with Hamming distances between sequences in MSAs. Therefore, MSA-based language models encode detailed phylogenetic relationships. This could aid them to separate coevolutionary signals encoding functional and structural constraints from phylogenetic correlations arising from historical contingency. To test this hypothesis, we generate synthetic MSAs, either without or with phylogeny, from Potts models trained on natural MSAs. We demonstrate that unsupervised contact prediction is indeed substantially more resilient to phylogenetic noise when using MSA Transformer versus inferred Potts models.

show abstract

Section: Discussionmentioning

confidence: 99%

Protein language models trained on multiple sequence alignments learn phylogenetic relationships

Lupo

Sgarbossa

Bitbol

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We have employed a GPU implementation of the MCMC methodology, which makes it computationally tractable without resorting to more approximate inverse inference methods. The MCMC algorithm implemented on GPUs has been previously used to infer accurate Potts models in [ 17 , 23 , 35 , 38 , 54 , 67 ].…”

Section: Methodsmentioning

confidence: 99%

Limits to detecting epistasis in the fitness landscape of HIV

2022

Self Cite

View full text Add to dashboard Cite

The rapid evolution of HIV is constrained by interactions between mutations which affect viral fitness. In this work, we explore the role of epistasis in determining the mutational fitness landscape of HIV for multiple drug target proteins, including Protease, Reverse Transcriptase, and Integrase. Epistatic interactions between residues modulate the mutation patterns involved in drug resistance, with unambiguous signatures of epistasis best seen in the comparison of the Potts model predicted and experimental HIV sequence “prevalences” expressed as higher-order marginals (beyond triplets) of the sequence probability distribution. In contrast, experimental measures of fitness such as viral replicative capacities generally probe fitness effects of point mutations in a single background, providing weak evidence for epistasis in viral systems. The detectable effects of epistasis are obscured by higher evolutionary conservation at sites. While double mutant cycles in principle, provide one of the best ways to probe epistatic interactions experimentally without reference to a particular background, we show that the analysis is complicated by the small dynamic range of measurements. Overall, we show that global pairwise interaction Potts models are necessary for predicting the mutational landscape of viral proteins.

show abstract

“…Potts model and threaded-energy calculation. Our Potts Hamiltonian was constructed from an MSA of protein kinase catalytic domains as previously described 67 . The Potts Hamiltonian ‫ܪ‬ሺܵሻ takes the form -…”

Section: Contact Frequency Differencesmentioning

confidence: 99%

“…2. An additional 20 type-I and type-I ½ ABFEs were calculated as part of our benchmarking procedure described in the Supplementary Information.Multiple Sequence Alignment (MSA) and Classification of Serine/Threonine vs Tyrosine Kinases.An MSA of 236,572 protein kinase catalytic domains with 259 columns was constructed as previously described 67. STKs and TKs were classified based on patterns of sequence conservation previously identified by Taylor and co-workers62 ; characteristic sequence features of TKs and STKs which form their respective phosphoacceptor binding pockets are found at the HRD+2 (Ala or Arg in TKs, Lys in STKs) and HRD+4 (Arg in TKs, variable in STKs) in the catalytic loop, as well as the APE-2 residue (Trp in TKs, variable in STKs) in the activation loop.…”

mentioning

confidence: 99%

Evolutionary Divergence in the Conformational Landscapes of Tyrosine vs Serine/Threonine Kinases

Gizzio

Thakur

Haldane

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Inactive conformations of protein kinase catalytic domains where the DFG motif has a "DFG-out" orientation and the activation loop is folded present a druggable binding pocket that is targeted by FDA-approved "type-II inhibitors" in the treatment of cancers. Tyrosine Kinases (TKs) typically show strong binding affinity with a wide spectrum of type-II inhibitors while Serine/Threonine Kinases (STKs) usually bind more weakly which we suggest here is due to differences in the folded to extended conformational equilibrium of the activation loop between TKs vs. STKs. To investigate this, we use sequence covariation analysis with a Potts Hamiltonian statistical energy model to guide absolute binding free-energy molecular dynamics simulations of 94 protein-ligand complexes. Using the calculated binding free energies together with experimental values, we estimated free-energy costs for the large-scale (~17-20Å) conformational change of the activation loop by an indirect approach, circumventing the very challenging problem of simulating the conformational change directly. We also used the Potts statistical potential to thread large sequence ensembles over active and inactive kinase states. The structure-based and sequence-based analyses are consistent; together they suggest TKs evolved to have free-energy penalties for the classical "folded activation loop" DFG-out conformation relative to the active conformation that is, on average, 4-6 kcal/mol smaller than the corresponding values for STKs. Potts statistical energy analysis suggests a molecular basis for this observation, wherein the activation loops of TKs are more weakly "anchored" against the catalytic loop motif in the active conformation, and form more stable substrate-mimicking interactions in the inactive conformation.

show abstract

The generative capacity of probabilistic protein sequence models

Cited by 30 publications

References 71 publications

Protein language models trained on multiple sequence alignments learn phylogenetic relationships

Protein language models trained on multiple sequence alignments learn phylogenetic relationships

Limits to detecting epistasis in the fitness landscape of HIV

Evolutionary Divergence in the Conformational Landscapes of Tyrosine vs Serine/Threonine Kinases

Contact Info

Product

Resources

About