Towards Locality-Aware Meta-Learning of Tail Node Embeddings on Networks

Liu, Zemin; Zhang, Wentao; Zhang, Xinming; Hoi, Steven C. H.

doi:10.1145/3340531.3411910

Cited by 46 publications

(33 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We hope that new models will more regularly include datasets such as Hetionet during their development phase. More generally, and taking cues from the field of GNNs [25,26,44], new methods could be be developed which consider how best to learn meaningful representations for low-degree entities.…”

Section: Discussionmentioning

confidence: 99%

“…The issue of non-uniform graph connectivity (typically in homogenous graphs) has begun to be studied in parallel by the field of Graph Neural Networks (GNN), where researchers have shown that models learn low-quality representations, thus making more incorrect predictions, for low-degree vertices [26,25,44]. This has also been explored in the context of homogenous graph representation learning [3] and for random walks [23,36].…”

Section: Previous Workmentioning

confidence: 99%

See 1 more Smart Citation

Implications of Topological Imbalance for Representation Learning on Biomedical Knowledge Graphs

Bonner,

Kirik,

Engkvist

et al. 2021

Preprint

View full text Add to dashboard Cite

Improving on the standard of care for a given disease is predicated on better treatments, which in turn relies on finding and developing new drugs. However, drug discovery is a complex, cross-disciplinary and costly process. Adoption of recently developed methods from machine learning has given rise to creation of drug discovery knowledge graphs which utilize the inherent interconnected nature of the domain. Graph-based modelling of the data, combined with knowledge graph embedding methods which aim to learn meaningful representations of biological entities, are promising as they provide a more intuitive representation of the domain and are suitable for inference tasks such as predicting missing links. In this context, one such example would be producing ranked lists of likely associated genes for a given disease, often referred to as target discovery. It is thus critical that these predictions are not only pertinent but also biologically meaningful.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Previous Workmentioning

confidence: 99%

Implications of Topological Imbalance for Representation Learning on Biomedical Knowledge Graphs

Bonner,

Kirik,

Engkvist

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Another subtype of meta-learning called hypernetwork [15,30] uses a secondary neural network to generate the weights for the target neural network, i.e., it learns to generate different weights conditioned on different input, instead of freezing the weights for all input after training in traditional neural networks. More recently, metalearning has also been adopted on graphs for few-shot learning, such as Meta-GNN [55], GFL [50], GPN [5], RALE [25] and meta-tail2vec [26], which is distinct from inductive semi-supervised node classification as further elaborated in Section 3.1. Hypernetworkbased approaches have also emerged, such as LGNN [24] that adapts GNN weights to different local contexts, and GNN-FiLM [3] that adapts to different relations in a relational graph.…”

Section: Related Workmentioning

confidence: 99%

Meta-Inductive Node Classification across Graphs

Wen

Liu

2021

Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Self Cite

View full text Add to dashboard Cite

Semi-supervised node classification on graphs is an important research problem, with many real-world applications in information retrieval such as content classification on a social network and query intent classification on an e-commerce query graph. While traditional approaches are largely transductive, recent graph neural networks (GNNs) integrate node features with network structures, thus enabling inductive node classification models that can be applied to new nodes or even new graphs in the same feature space. However, inter-graph differences still exist across graphs within the same domain. Thus, training just one global model (e.g., a state-of-the-art GNN) to handle all new graphs, whilst ignoring the inter-graph differences, can lead to suboptimal performance.In this paper, we study the problem of inductive node classification across graphs. Unlike existing one-model-fits-all approaches, we propose a novel meta-inductive framework called MI-GNN to customize the inductive model to each graph under a meta-learning paradigm. That is, MI-GNN does not directly learn an inductive model; it learns the general knowledge of how to train a model for semi-supervised node classification on new graphs. To cope with the differences across graphs, MI-GNN employs a dual adaptation mechanism at both the graph and task levels. More specifically, we learn a graph prior to adapt for the graph-level differences, and a task prior to adapt for the task-level differences conditioned on a graph. Extensive experiments on five real-world graph collections demonstrate the effectiveness of our proposed model. CCS CONCEPTS• Computing methodologies → Learning latent representations; • Information systems → Data mining.

show abstract

“…This problem is often challenging in practice because the degree distributions of most graphs follow a power law distribution and there are many nodes with very few connections. Liu et al [Liu+20] address this issue by applying meta-learning to the problem of node embedding of graphs. They set up a regression problem with a common prior to learn the node embeddings.…”

Section: Meta-learning Applied To Graph Problemsmentioning

confidence: 99%

Meta-Learning with Graph Neural Networks: Methods and Applications

Mandal¹,

Medya²,

Uzzi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Graph Neural Networks (GNNs), a generalization of deep neural networks on graph data have been widely used in various domains, ranging from drug discovery to recommender systems. However, GNNs on such applications are limited when there are few available samples. Meta-learning has been an important framework to address the lack of samples in machine learning, and in recent years, the researchers have started to apply meta-learning to GNNs. In this work, we provide a comprehensive survey of different meta-learning approaches involving GNNs on various graph problems showing the power of using these two approaches together. We categorize the literature based on proposed architectures, shared representations, and applications. Finally, we discuss several exciting future research directions and open problems.

show abstract

Towards Locality-Aware Meta-Learning of Tail Node Embeddings on Networks

Cited by 46 publications

References 29 publications

Implications of Topological Imbalance for Representation Learning on Biomedical Knowledge Graphs

Implications of Topological Imbalance for Representation Learning on Biomedical Knowledge Graphs

Meta-Inductive Node Classification across Graphs

Meta-Learning with Graph Neural Networks: Methods and Applications

Contact Info

Product

Resources

About