Protein Representation Learning by Geometric Structure Pretraining

Zhang, Zuobai; Xu, Minghao; Jamasb, Arian R.; Chenthamarakshan, Vijil; Lozano, Aurélie; Das, Payel; Tang, J.

doi:10.48550/arxiv.2203.06125

Cited by 40 publications

(87 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A protein chain is essentially a sequence of amino acids, and such sequential information is shown to be crucial to determine protein functions. Hence, we follow existing studies [24,64] and integrate the sequential information in edge features. Specifically, for each edge ij that is from node i to node j, the edge feature includes an embedding of the sequential distance j − i.…”

Section: Amino Acid Levelmentioning

confidence: 99%

“…Importantly, there exist hierarchical relations among different levels. Existing methods for protein representation learning either ignore hierarchical relations within proteins [26,33,20,64], or suffer from excessive computational complexity [25,21]. In this work, Figure 1: Illustration of hierarchical representations for protein structures.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Hierarchical Protein Representations via Complete 3D Graph Networks

Wang¹,

Liu²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

We consider representation learning for proteins with 3D structures. We build 3D graphs based on protein structures and develop graph networks to learn their representations. Depending on the levels of details that we wish to capture, protein representations can be computed at different levels, e.g., the amino acid, backbone, or all-atom levels. Importantly, there exist hierarchical relations among different levels. In this work, we propose to develop a novel hierarchical graph network, known as ProNet, to capture the relations. Our ProNet is very flexible and can be used to compute protein representations at different levels of granularity. We show that, given a base 3D graph network that is complete, our ProNet representations are also complete at all levels. To close the loop, we develop a complete and efficient 3D graph network to be used as a base model, making our ProNet complete. We conduct experiments on multiple downstream tasks. Results show that ProNet outperforms recent methods on most datasets. In addition, results indicate that different downstream tasks may require representations at different levels. Our code is available as part of the DIG library (https://github.com/divelab/DIG). * Equal contribution † Equal senior contribution Preprint. Under review.

show abstract

Section: Amino Acid Levelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learning Hierarchical Protein Representations via Complete 3D Graph Networks

Wang¹,

Liu²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Hermosilla and Ropinski [34] uses contrastive learning for representation learning of 3D protein structures from the perspective of sub-structures. Apart from that, Zhang et al [98] combines a multi-view contrastive learning and a self-prediction learning to encode geometric features of proteins. Then these semantic representations learned from SSL are utilized for downstream tasks including structure classification [35], model quality assessment [4], and function prediction [30].…”

Section: Related Workmentioning

confidence: 99%

“…They take advantage of millions of sequences to pre-train protein encoders via SSL [18,13,96]. In addition, some methods [34,98] start to directly capture the available protein structural information, which is proven to be the determinants of protein functio. Despite the fruitful progress, none of prior work considers the exploitation of temporal sequences of protein structures.…”

mentioning

confidence: 99%

Pre-training of Equivariant Graph Matching Networks with Conformation Flexibility for Drug Binding

Wu¹,

Jiang²,

Jin³

et al. 2022

Preprint

View full text Add to dashboard Cite

The latest biological findings discover that the motionless 'lock-and-key' theory is no longer applicable and the flexibility of both the receptor and ligand plays a significant role in helping understand the principles of the binding affinity prediction. Based on this mechanism, molecular dynamics (MD) simulations have been invented as a useful tool to investigate the dynamical properties of this molecular system. However, the computational expenditure prohibits the growth of reported protein trajectories. To address this insufficiency, we present a novel spatial-temporal pre-training protocol, PretrainMD, to grant the protein encoder the capacity to capture the time-dependent geometric mobility along MD trajectories. Specifically, we introduce two sorts of self-supervised learning tasks: an atom-level denoising generative task and a protein-level snapshot ordering task. We validate the effectiveness of PretrainMD through the PDBbind dataset for both linear-probing and fine-tuning. Extensive experiments show that our PretrainMD exceeds most state-of-the-art methods and achieves comparable performance. More importantly, through visualization we discover that the learned representations by pre-training on MD trajectories without any label from the downstream task follow similar patterns of the magnitude of binding affinities. This strongly aligns with the fact that the motion of the interactions of protein and ligand maintains the key information of their binding. Our work provides a promising perspective of self-supervised pre-training for protein representations with very fine temporal * The corresponding authors.Preprint. Under review.

show abstract

“…While most of the sequence models rely on the transformer architecture, CARP [57] finds that CNNs can achieve competitive results with much fewer parameters and runtime costs. Recently, GearNet [64] explores the potential of 3D structural pre-training from the perspective of masked prediction and contrastive learning. We also summarize the molecule pretraining models in Table .1.…”

Section: Infoncementioning

confidence: 99%

CoSP: Co-supervised pretraining of pocket and ligand

Gao¹,

Tan²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Can we inject the pocket-ligand interaction knowledge into the pre-trained model and jointly learn their chemical space? Pretraining molecules and proteins has attracted considerable attention in recent years, while most of these approaches focus on learning one of the chemical spaces and lack the injection of biological knowledge. We propose a co-supervised pretraining (CoSP) framework to simultaneously learn 3D pocket and ligand representations. We use a gated geometric message passing layer to model both 3D pockets and ligands, where each node's chemical features, geometric position and orientation are considered. To learn biological meaningful embeddings, we inject the pocket-ligand interaction knowledge into the pretraining model via contrastive loss. Considering the specificity of molecules, we further propose a chemical similarity-enhanced negative sampling strategy to improve the contrastive learning performance. Through extensive experiments, we conclude that CoSP can achieve competitive results in pocket matching, molecule property predictions, and virtual screening.Preprint. Under review.

show abstract

Protein Representation Learning by Geometric Structure Pretraining

Cited by 40 publications

References 0 publications

Learning Hierarchical Protein Representations via Complete 3D Graph Networks

Learning Hierarchical Protein Representations via Complete 3D Graph Networks

Pre-training of Equivariant Graph Matching Networks with Conformation Flexibility for Drug Binding

CoSP: Co-supervised pretraining of pocket and ligand

Contact Info

Product

Resources

About