Motif-Driven Contrastive Learning of Graph Representations

Zhang, Shichang; Hu, Ziniu; Subramonian, Arjun; Sun, Yizhou

doi:10.48550/arxiv.2012.12533

Cited by 16 publications

(33 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, GCA is focused on network data and not suitable for molecular graphs. Instead of focusing on augmentation views, MICRO-Graph [46] proposed to contrast based on sub-graphs (motifs). GCC [24] proposed to use random walk to generate subgraphs and contrast between them.…”

Section: Related Workmentioning

confidence: 99%

MoCL: Data-driven Molecular Fingerprint via Knowledge-aware Contrastive Learning from Molecular Graph

Sun¹,

Xing²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent years have seen a rapid growth of utilizing graph neural networks (GNNs) in the biomedical domain for tackling drug-related problems. However, like any other deep architectures, GNNs are data hungry. While requiring labels in real world is often expensive, pretraining GNNs in an unsupervised manner has been actively explored. Among them, graph contrastive learning, by maximizing the mutual information between paired graph augmentations, has been shown to be effective on various downstream tasks. However, the current graph contrastive learning framework has two limitations. First, the augmentations are designed for general graphs and thus may not be suitable or powerful enough for certain domains. Second, the contrastive scheme only learns representations that are invariant to local perturbations and thus does not consider the global structure of the dataset, which may also be useful for downstream tasks. Therefore, in this paper, we study graph contrastive learning in the context of biomedical domain, where molecular graphs are present. We propose a novel framework called MoCL, which utilizes domain knowledge at both local-and global-level to assist representation learning. The local-level domain knowledge guides the augmentation process such that variation is introduced without changing graph semantics. The global-level knowledge encodes the similarity information between graphs in the entire dataset and helps to learn representations with richer semantics. The entire model is learned through a double contrast objective. We evaluate MoCL on various molecular datasets under both linear and semi-supervised settings and results show that MoCL achieves state-of-the-art performance.

show abstract

Section: Related Workmentioning

confidence: 99%

MoCL: Data-driven Molecular Fingerprint via Knowledge-aware Contrastive Learning from Molecular Graph

Sun¹,

Xing²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Furthermore, [39,28,41] proposed to use the mutual information maximization as the optimization objective for GNNs. Recently, more self-supervised tasks for GNNs have been proposed [14,15,52,51,32,10,54,30]. Based on how the self-supervised tasks are constructed, these models can be classified into two categories, namely contrastive models and predictive models.…”

Section: Self-supervised Learning Of Graphsmentioning

confidence: 99%

“…Based on how the self-supervised tasks are constructed, these models can be classified into two categories, namely contrastive models and predictive models. Contrastive models try to generate informative and diverse views from data instances and perform node-to-context [14], node-to-graph [39] or motif-to-graph [54] contrastive learning. On the other hand, predictive models are trained in a supervised fashion, where the labels are generated based on certain properties of the input graph data, i.e., node attributes [32], or by selecting certain parts of the graph [15,32].…”

Section: Self-supervised Learning Of Graphsmentioning

confidence: 99%

See 1 more Smart Citation

Motif-based Graph Self-Supervised Learning for Molecular Property Prediction

Zhang¹,

Liu²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Predicting molecular properties with data-driven methods has drawn much attention in recent years. Particularly, Graph Neural Networks (GNNs) have demonstrated remarkable success in various molecular generation and prediction tasks. In cases where labeled data is scarce, GNNs can be pre-trained on unlabeled molecular data to first learn the general semantic and structural information before being finetuned for specific tasks. However, most existing self-supervised pre-training frameworks for GNNs only focus on node-level or graph-level tasks. These approaches cannot capture the rich information in subgraphs or graph motifs. For example, functional groups (frequently-occurred subgraphs in molecular graphs) often carry indicative information about the molecular properties. To bridge this gap, we propose Motifbased Graph Self-supervised Learning (MGSSL) by introducing a novel selfsupervised motif generation framework for GNNs. First, for motif extraction from molecular graphs, we design a molecule fragmentation method that leverages a retrosynthesis-based algorithm BRICS and additional rules for controlling the size of motif vocabulary. Second, we design a general motif-based generative pre-training framework in which GNNs are asked to make topological and label predictions. This generative framework can be implemented in two different ways, i.e., breadth-first or depth-first. Finally, to take the multi-scale information in molecular graphs into consideration, we introduce a multi-level self-supervised pre-training. Extensive experiments on various downstream benchmark tasks show that our methods outperform all state-of-the-art baselines.

show abstract

“…The current lack of unified evaluation criterion on graph embedding-based recommendation and conventional recommendation will lead to longstanding discussions on these controversies in the future, involving expanded perspectives from accuracy, scalability, extensibility, and explainability, as well as participated by interdisciplinary researchers ranging from mathematicians to data scientists. Developing both graph embedding-based recommendation and conventional recommendation is not contradictory, for the methods of analyzing graph topological characteristics behind conventional recommendation can inspire graph embedding-based recommendation in the utilization of such as subgraphs [64], motifs [65][66][67], and neighborhood [68][69][70] to promote embedding explainability [39] and recommendation performance. Meanwhile, graph embedding-based recommendation has pioneered novel recommendation scenarios including conversational recommender system (CRS) [71] and news recommendation [72], providing more promising application prospects for conventional recommendation.…”

Section: Introductionmentioning

confidence: 99%

Recommender systems based on graph embedding techniques: A comprehensive review

Deng

2021

Preprint

View full text Add to dashboard Cite

Recommender systems, a pivotal tool to alleviate the information overload problem, aim to predict user's preferred items from millions of candidates by analyzing observed user-item relations. As for tackling the sparsity and cold start problems encountered by recommender systems, uncovering hidden (indirect) user-item relations by employing side information and knowledge to enrich observed information for the recommendation has been proven promising recently; and its performance is largely determined by the scalability of recommendation models in the face of the high complexity and large scale of side information and knowledge. Making great strides towards efficiently utilizing complex and large-scale data, research into graph embedding techniques is a major topic. Equipping recommender systems with graph embedding techniques contributes to outperforming the conventional recommendation implementing directly based on graph topology analysis and has been widely studied these years. This article systematically retrospects graph embedding-based recommendation from embedding techniques for bipartite graphs, general graphs, and knowledge graphs, and proposes a general design pipeline of that. In addition, comparing several representative graph embedding-based recommendation models with the most common-used conventional recommendation models, on simulations, manifests that the conventional models overall outperform the graph embedding-based ones in predicting implicit user-item interactions, revealing the relative weakness of graph embedding-based recommendation in these tasks. To foster future research, this article proposes constructive suggestions on making a trade-off between graph embedding-based recommendation and the conventional recommendation in different tasks as well as some open questions.

show abstract

Motif-Driven Contrastive Learning of Graph Representations

Cited by 16 publications

References 18 publications

MoCL: Data-driven Molecular Fingerprint via Knowledge-aware Contrastive Learning from Molecular Graph

MoCL: Data-driven Molecular Fingerprint via Knowledge-aware Contrastive Learning from Molecular Graph

Motif-based Graph Self-Supervised Learning for Molecular Property Prediction

Recommender systems based on graph embedding techniques: A comprehensive review

Contact Info

Product

Resources

About