Physics-Inspired Protein Encoder Pre-Training via Siamese Sequence-Structure Diffusion Trajectory Prediction

Zhang, Zuobai; Xu, Minghao; Lozano, Aurélie; Chenthamarakshan, Vijil; Das, Payel; Tang, J.

doi:10.48550/arxiv.2301.12068

Cited by 4 publications

(4 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the geometric encoder is able to utilize labeled protein complex structures in ΔΔ G bind datasets, training on a limited set of mutation data could result in overfitting and poor generalization. To address this problem, we further propose a self-supervised pretraining task to exploit large amounts of unlabeled protein structures in PDB [21, 22]. In the pretraining stage, the encoder is trained to model the distribution of these native protein structures via noise contrastive estimation [23].…”

Section: Resultsmentioning

confidence: 99%

“…Although GearBind can be trained from scratch on labeled ΔΔ G bind datasets, it could suffer from overfitting or poor generalization if the training data size is limited. To address this problem, we propose a self-supervised pretraining task to exploit large-scale unlabeled protein structures in CATH [22, 23]. In the pretraining stage, the encoder is trained to model the distribution of the native protein structures via noise contrastive estimation [24].…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Pretrainable Geometric Graph Neural Network for Antibody Affinity Maturation

Cai,

Zhang,

Wang

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

In the realm of antibody therapeutics development, increasing the binding affinity of an antibody to its target antigen is a crucial task. This paper presents GearBind, a pretrainable deep neural network designed to be effective for in silico affinity maturation. Leveraging multi-level geometric message passing alongside contrastive pretraining on protein structural data, GearBind capably models the complex interplay of atom-level interactions within protein complexes, surpassing previous state-of-the-art approaches on SKEMPI v2 in terms of Pearson correlation, mean absolute error (MAE) and root mean square error (RMSE). In silico experiments elucidate that pretraining helps GearBind become sensitive to mutation-induced binding affinity changes and reflective of amino acid substitution tendency. Using an ensemble model based on pretrained GearBind, we successfully optimize the affinity of CR3022 to the spike (S) protein of the SARS-CoV-2 Omicron strain. Our strategy yields a high success rate with up to 17-fold affinity increase. GearBind proves to be an effective tool in narrowing the search space for in vitro antibody affinity maturation, underscoring the utility of geometric deep learning and adept pre-training in macromolecule interaction modeling.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Pretrainable Geometric Graph Neural Network for Antibody Affinity Maturation

Cai,

Zhang,

Wang

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Gligorijević et al (2021); Zhang et al (2022); Xu et al (2022) learn residues from a local part of protein structures. Jing et al (2020); Zhang et al (2023) try to capture atomic structure knowledge in proteins. We develop ms -ESM based on ESM.…”

Section: Related Workmentioning

confidence: 99%

ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular Modeling

Zheng,

Long,

et al. 2024

Preprint

View full text Add to dashboard Cite

Protein language models have demonstrated significant potential in the field of protein engineering. However, current protein language models primarily operate at the residue scale, which limits their ability to provide information at the atom level. This limitation prevents us from fully exploiting the capabilities of protein language models for applications involving both proteins and small molecules. In this paper, we propose ms-ESM (multi-scale ESM), a novel approach that enables multi-scale unified molecular modeling. ms-ESM achieves this by pre-training on multi-scale code-switch protein sequences and utilizing a multi-scale position encoding to capture relationships among residues and atoms. Experimental results indicate that ms-ESM surpasses previous methods in protein-molecule tasks, demonstrating the full utilization of protein language models. Further investigations reveal that through unified molecular modeling, ms-ESM not only gains molecular knowledge but also retains its understanding of proteins.

show abstract

“…Graph neural networks (GNNs) have shown remarkable promise in modeling proteins and have been successfully applied to various protein tasks, including fold classification 40,41 , property prediction [40][41][42] , fixed backbone sequence design 27,[43][44][45][46] , and PSCP 24,[27][28][29] . With the rationale that the specific side chain conformations are primarily dependent upon the local environment of the amino acid, we decided to model the PSCP problem with a GNN, wherein each residue is modeled as a node and is connected to its 𝑘 nearest neighbors.…”

Section: Architectural Considerationsmentioning

confidence: 99%

Invariant point message passing for protein side chain packing

Randolph

Kuhlman

2023

Preprint

View full text Add to dashboard Cite

Protein side chain packing (PSCP) is a fundamental problem in the field of protein engineering, as high confidence and low energy conformations of amino acid side chains are crucial for understanding (and designing) protein folding, protein-protein interactions, and protein-ligand interactions. Traditional PSCP methods (such as the Rosetta Packer) often rely on a library of discrete side chain conformations, or rotamers, and a forcefield to guide the structure to low energy conformations. Recently, deep learning (DL) based methods (such as DLPacker, AttnPacker, and DiffPack) have demonstrated state-of-the-art predictions and speed in the PSCP task. Building off the success of graph and message passing neural networks for protein modeling, we present the Protein Invariant Point Packer (PIPPack) which effectively processes local structural and sequence information to produce realistic, idealized side chain coordinates using 𝜒-angle distribution predictions and geometry-aware invariant point message passing (IPMP). To demonstrate its broad applicability to protein-related tasks, IPMP was additionally incorporated in a fixed backbone protein design method, which enabled the generation of more native-like sequences than common message passing schemes. On a test set of ∼1,400 high-quality protein chains, PIPPack performs competitively with other state-of-the-art PSCP methods in rotamer recovery and per-residue RMSDs but is significantly faster.

show abstract

Physics-Inspired Protein Encoder Pre-Training via Siamese Sequence-Structure Diffusion Trajectory Prediction

Cited by 4 publications

References 46 publications

Pretrainable Geometric Graph Neural Network for Antibody Affinity Maturation

Pretrainable Geometric Graph Neural Network for Antibody Affinity Maturation

ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular Modeling

Invariant point message passing for protein side chain packing

Contact Info

Product

Resources

About