HelixFold-Single: MSA-free Protein Structure Prediction by Using Protein Language Model as an Alternative

Abstract

AI-based protein structure prediction pipelines, such as AlphaFold2, have achieved near-experimental accuracy. These advanced pipelines mainly rely on Multiple Sequence Alignments (MSAs) and templates as inputs to learn the co-evolution information from the homologous sequences. Nonetheless, searching MSAs and templates from protein databases is time-consuming, usually taking dozens of minutes. Consequently, we attempt to explore the limits of fast protein structure prediction by using only primary sequences of proteins. HelixFold-Single is proposed to combine a largescale protein language model with the superior geometric learning capability of AlphaFold2. Our proposed method, HelixFold-Single, first pre-trains a large-scale protein language model (PLM) with thousands of millions of primary sequences utilizing the self-supervised learning paradigm, which will be used as an alternative to MSAs and templates for learning the co-evolution information. Then, by combining the pre-trained PLM and the essential components of AlphaFold2, we obtain an end-to-end differentiable model to predict the 3D coordinates of atoms from only the primary sequence. HelixFold-Single is validated in datasets CASP14 and CAMEO, achieving competitive accuracy with the MSA-based methods on the targets with large homologous families. Furthermore, HelixFold-Single consumes much less time than the mainstream pipelines for protein structure prediction, demonstrating its potential in tasks requiring many predictions. The code of HelixFold-Single is available at https://github.com/PaddlePaddle/PaddleHelix/tree/ dev/apps/protein_folding/helixfold-single, and we also provide stable web services on https://paddlehelix.baidu.com/app/drug/protein-single/forecast.

tFold-Ab: Fast and Accurate Antibody Structure Prediction without Sequence Homologs

“…Evaluation. Baseline methods include antibody-specific structure prediction (ABodyBuilder [19], DeepAb [34], ABlooper [1], NanoNet [8] and IgFold [31]) and general protein structure prediction, either MSA-based (AlphaFold [15] and AlphaFold-Multimer [10]) or MSA-free (HelixFold-Single [11], ESMFold [20], and OmegaFold [40]).…”

“…These models mainly use a attention-based deep neural network to capture long-range inter-residue relationship and coevolutionary information encoded in the sequence. Previous work [9,28,23,20] has shown that with small-scale supervised training for downstream tasks, PLMs can capture some functional and structure properties of proteins, including secondary structure, binding residues [21], tertiary contact and protein structure [20,40,11]. Some work [29] also extends the PLMs to model a set of aligned sequences in a MSA using axial attention.…”

“…However, both methods heavily rely on sequence homology inputs, which can be time-consuming for large-scale sequence databases. Other methods [11,40] turn to pre-trained language models (PLMs) to circumvent the costly MSA search, but their precision has yet to improve due to the lack of explicit sequence homologs. Yet, the performance of such end-to-end prediction methods is not extensively verified for antibody structures, especially for CDR loops and multimer sequence inputs.…”

So ManyFolds, So Little Time: Efficient Protein Structure Prediction With pLMs and MSAs

“…without MSAs or templates (model_5_ptm zeroing and masking MSA features). We compare to recent pLM-based models trained for protein structure prediction from sequence alone -specifically OmegaFold [5] and HelixFold-Single [13], in both cases using the official publicly available code and checkpoints. The recent ESMFold [12] model (which is a minimally modified AlphaFold Evoformer and Structure Module operating on ESM-2 embeddings) does not yet have a public release, and so we cannot report a direct comparison given the different validation sets between their paper and ours.…”

“…An appealing approach is therefore to replace explicit representations of external sequence/structural databases (i.e. MSAs/templates) with pLM embeddings, which has been the foundation of recent models that predict 3D structures from protein sequence alone [5,12,13].…”