Adaptive Universal Generalized PageRank Graph Neural Network

Chien, Eli; Peng, Jianhao; Li, Pan; Milenković, Olgica

doi:10.48550/arxiv.2006.07988

Cited by 24 publications

(41 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are also works (Zhu et al, 2020b;Bo et al, 2021;Chien et al, 2020;Yan et al, 2021;Suresh et al, 2021;Pei et al, 2020;Dong et al, 2021;Lim et al, 2021;Yang et al, 2021;Luan et al, 2021;Zhu et al, 2020a;Liu et al, 2021) that extend GNNs to heterophilous graphs. Some methods propose to leverage both low-pass and high-pass convolutional filters in neighborhood aggregation.…”

Section: Related Workmentioning

confidence: 99%

“…To generalize GNNs to heterophilous graphs, some recent works (Zhu et al, 2020b;Bo et al, 2021;Chien et al, 2020) have been proposed to leverage high-pass convolutional filters and multi-hop neighbors to address the heterophily issue. On the one hand, the high-pass filters can be used to push away a node's feature vector from its neighbors' while the low-pass filters used by traditional GNNs do the opposite.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Finding Global Homophily in Graph Neural Networks When Meeting Heterophily

Li¹,

Renyu²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

We investigate graph neural networks on graphs with heterophily. Some existing methods amplify a node's neighborhood with multi-hop neighbors to include more nodes with homophily. However, it is a significant challenge to set personalized neighborhood sizes for different nodes. Further, for other homophilous nodes excluded in the neighborhood, they are ignored for information aggregation. To address these problems, we propose two models GloGNN and GloGNN++, which generate a node's embedding by aggregating information from global nodes in the graph. In each layer, both models learn a coefficient matrix to capture the correlations between nodes, based on which neighborhood aggregation is performed. The coefficient matrix allows signed values and is derived from an optimization problem that has a closed-form solution. We further accelerate neighborhood aggregation and derive a linear time complexity. We theoretically explain the models' effectiveness by proving that both the coefficient matrix and the generated node embedding matrix have the desired grouping effect. We conduct extensive experiments to compare our models against 11 other competitors on 15 benchmark datasets in a wide range of domains, scales and graph heterophilies. Experimental results show that our methods achieve superior performance and are also very efficient.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Finding Global Homophily in Graph Neural Networks When Meeting Heterophily

Li¹,

Renyu²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The LGC algorithm arising from our analysis smooths features of nodes over the graph, using the output of label propagation on the features as input to a linear model. The idea of feature smoothing arises in many contexts for understanding GNNs [27,37,38,39], although some studies claim that "over-smoothing" is problematic [40,41,42]. Our model provides a statistical view of feature smoothing and a principled way to find an optimal smoothing level, which we can interpret as a balance between the effects of homophily and noise.…”

Section: The Present Workmentioning

confidence: 99%

“…Equations (2.22) and (2.24) both highlight how each column of X is smooth over the graph, and the parameter ω controls the amount of smoothing, as dictated by the generative data model. From the perspective of graph signal processing, similar smoothing has been viewed as a low-pass filter for features [56,36], and feature smoothing is a heuristic for several graph-based learning methods [27,37,38,39]. Our model puts these ideas on firm ground.…”

mentioning

confidence: 99%

A Unifying Generative Model for Graph Learning Algorithms: Label Propagation, Graph Convolutions, and Combinations

Jia¹,

Benson²

2021

Preprint

View full text Add to dashboard Cite

Semi-supervised learning on graphs is a widely applicable problem in network science and machine learning. Two standard algorithms -label propagation and graph neural networks -both operate by repeatedly passing information along edges, the former by passing labels and the latter by passing node features, modulated by neural networks. These two types of algorithms have largely developed separately, and there is little understanding about the structure of network data that would make one of these approaches work particularly well compared to the other or when the approaches can be meaningfully combined. Here, we develop a Markov random field model for the data generation process of node attributes, based on correlations of attributes on and between vertices, that motivates and unifies these algorithmic approaches. We show that label propagation, a linearized graph convolutional network, and their combination can all be derived as conditional expectations under our model. In addition, the data model highlights problems with existing graph neural networks (and provides solutions), serves as a rigorous statistical framework for understanding issues such as over-smoothing, creates a testbed for evaluating inductive learning performance, and provides a way to sample graphs attributes that resemble empirical data. We also find that a new algorithm derived from our data generation model, which we call a Linear Graph Convolution, performs extremely well in practice on empirical data, and provide theoretical justification for why this is the case.

show abstract

“…The central part of our algorithm is a new random-walk-based pooling mechanism called WalkPool which may be interepreted as a learnable version of topological heuristics (Figure 1). Our algorithm is universal in the sense of Chien et al (2020) in that it models both heterophilic and homophilic graphs unlike hardcoded heuristics or standard GNNs which work better on homophilic graphs. WalkPool achieves state-of-the-art link prediction results on all common benchmark datasets, sometimes by a significant margin, even on datasets like USAir (Batagelj & Mrvar, 2006) where the previous state of the art is as high as 97%.…”

Section: Completed Graphmentioning

confidence: 99%

Neural Link Prediction with Walk Pooling

Pan¹,

Shi²

2021

Preprint

View full text Add to dashboard Cite

Graph neural networks achieve high accuracy in link prediction by jointly leveraging graph topology and node attributes. Topology, however, is represented indirectly; state-ofthe-art methods based on subgraph classification label nodes with distance to the target link, so that, although topological information is present, it is tempered by pooling. This makes it challenging to leverage features like loops and motifs associated with network formation mechanisms. We propose a link prediction algorithm based on a new pooling scheme called WalkPool. WalkPool combines the expressivity of topological heuristics with the feature-learning ability of neural networks. It summarizes a putative link by random walk probabilities of adjacent paths. Instead of extracting transition probabilities from the original graph, it computes the transition matrix of a "predictive" latent graph by applying attention to learned features; this may be interpreted as feature-sensitive topology fingerprinting. WalkPool can leverage unsupervised node features or be combined with GNNs and trained end-to-end. It outperforms state-of-the-art methods on all common link prediction benchmarks, both homophilic and heterophilic, with and without node attributes. Applying WalkPool to a set of unsupervised GNNs significantly improves prediction accuracy, suggesting that it may be used as a general-purpose graph pooling scheme.

show abstract

Adaptive Universal Generalized PageRank Graph Neural Network

Cited by 24 publications

References 0 publications

Finding Global Homophily in Graph Neural Networks When Meeting Heterophily

Finding Global Homophily in Graph Neural Networks When Meeting Heterophily

A Unifying Generative Model for Graph Learning Algorithms: Label Propagation, Graph Convolutions, and Combinations

Neural Link Prediction with Walk Pooling

Contact Info

Product

Resources

About