Protein language models trained on multiple sequence alignments learn phylogenetic relationships

Lupo, Umberto; Sgarbossa, Damiano; Bitbol, Anne-Florence

doi:10.1101/2022.03.29.486219

Cited by 6 publications

(16 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We showed in another work that they are even more resilient to phylogenetic noise than Potts model [43]. Taken together, the present results and those of [43] demonstrate the crucial importance of disentangling phylogenetic correlations from functional ones for contact prediction performance. While phylogenetic correlations are an issue for structure prediction, they are nevertheless an important and helpful signal for the inference of interaction partners among the paralogs of two protein families [32, 44, 45].…”

Section: Discussionsupporting

confidence: 80%

Impact of phylogeny on structural contact inference from protein sequence data

Dietler

Lupo

Bitbol

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Local and global inference methods have been developed to infer structural contacts from multiple sequence alignments of homologous proteins. They rely on correlations in amino-acid usage at contacting sites. Because homologous proteins share a common ancestry, their sequences also feature phylogenetic correlations, which can impair contact inference. We investigate this effect by generating controlled synthetic data from a minimal model where the importance of contacts and of phylogeny can be tuned. We demonstrate that global inference methods, specifically Potts models, are more resilient to phylogenetic correlations than local methods, based on covariance or mutual information. This holds whether or not phylogenetic corrections are used, and may explain the success of global methods. We analyse the roles of selection strength and of phylogenetic relatedness. We show that sites that mutate early in the phylogeny yield false positive contacts. We consider natural data and realistic synthetic data, and our findings generalise to these cases. Our results highlight the impact of phylogeny on contact prediction from protein sequences and illustrate the interplay between the rich structure of biological data and inference.

show abstract

Section: Discussionsupporting

confidence: 80%

Impact of phylogeny on structural contact inference from protein sequence data

Dietler

Lupo

Bitbol

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The coevolution-driven approach was recently experimentally validated in the case of bmDCA Potts models, which capture pairwise coevolution patterns in MSAs [13], and for variational autoencoders [14,15]. Protein language models trained on multiple sequence alignments provide state-of-the-art unsupervised contact prediction and are able to capture coevolutionary patterns in their tied row attentions [25], and capture phylogenetic relationships in column attentions [53]. This makes them ideal candidates to generate new protein sequences from given families.…”

Section: Discussionmentioning

confidence: 99%

Generative power of a protein language model trained on multiple sequence alignments

Sgarbossa¹,

Lupo²,

Bitbol³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Computational models starting from large ensembles of evolutionarily related protein sequences capture a representation of protein families and learn constraints associated to protein structure and function. They thus open the possibility for generating novel sequences belonging to protein families. Protein language models trained on multiple sequence alignments, such as MSA Transformer, are highly attractive candidates to this end. We propose and test an iterative method that directly uses the masked language modeling objective to generate sequences using MSA Transformer. We demonstrate that the resulting sequences generally score better than those generated by Potts models, and even than natural sequences, for homology, coevolution and structure-based measures. Moreover, MSA Transformer better reproduces the higher-order statistics and the distribution of sequences in sequence space of natural data than Potts models, although Potts models better reproduce first-and second-order statistics. MSA Transformer is thus a strong candidate for protein sequence generation and protein design.

show abstract

“…To sample independent equilibrium sequences from Potts models, we used the strategy described in Ref. [53]. Specifically, we fitted Potts models on each of our natural MSAs using bmDCA [12] (https://github.com/ranganathanlab/bmDCA) with its default hyperparameters.…”

Section: Sampling Sequences From Potts Modelsmentioning

confidence: 99%

“…These alignments are the same as in Ref. [53], except that we discarded PF13354 (Beta-lactamase2) because of its smaller depth.…”

Section: Datasetsmentioning

confidence: 99%

Generative power of a protein language model trained on multiple sequence alignments

Sgarbossa

Lupo

Bitbol

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Computational models starting from large ensembles of evolutionarily related protein sequences capture a representation of protein families and learn constraints associated to protein structure and function. They thus open the possibility for generating novel sequences belonging to protein families. Protein language models trained on multiple sequence alignments, such as MSA Transformer, are highly attractive candidates to this end. We propose and test an iterative method that directly uses the masked language modeling objective to generate sequences using MSA Transformer. We demonstrate that the resulting sequences generally score better than those generated by Potts models, and even than natural sequences, for homology, coevolution and structure-based measures. Moreover, MSA Transformer better reproduces the higher-order statistics and the distribution of sequences in sequence space of natural data than Potts models, although Potts models better reproduce first- and second-order statistics. MSA Transformer is thus a strong candidate for protein sequence generation and protein design.

show abstract

Protein language models trained on multiple sequence alignments learn phylogenetic relationships

Cited by 6 publications

References 62 publications

Impact of phylogeny on structural contact inference from protein sequence data

Impact of phylogeny on structural contact inference from protein sequence data

Generative power of a protein language model trained on multiple sequence alignments

Generative power of a protein language model trained on multiple sequence alignments

Contact Info

Product

Resources

About