A Unifying Generative Model for Graph Learning Algorithms: Label Propagation, Graph Convolutions, and Combinations

Jia, Junteng; Benson, Austin R.

doi:10.48550/arxiv.2101.07730

Cited by 10 publications

(16 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The aggregation process can be usually understood as feature smoothing [20,21,16,42]. Hence, several recent works claim [41,40,4], assume [12,35,38] or remark upon [1,22,14] GNN models homophily-reliance or unsuitability in capturing heterophily.…”

Section: Related Workmentioning

confidence: 99%

Is Homophily a Necessity for Graph Neural Networks?

Ma¹,

Liu²,

Shah³

et al. 2021

Preprint

View full text Add to dashboard Cite

Graph neural networks (GNNs) have shown great prowess in learning representations suitable for numerous graph-based machine learning tasks. When applied to semi-supervised node classification, GNNs are widely believed to work well due to the homophily assumption ("like attracts like"), and fail to generalize to heterophilous graphs where dissimilar nodes connect. Recent works design new architectures to overcome such heterophily-related limitations, citing poor baseline performance and new architecture improvements on a few heterophilous graph benchmark datasets as evidence for this notion. In our experiments, we empirically find that standard graph convolutional networks (GCNs) can actually achieve better performance than such carefully designed methods on some commonly used heterophilous graphs. This motivates us to reconsider whether homophily is truly necessary for good GNN performance. We find that this claim is not quite true, and in fact, GCNs can achieve strong performance on heterophilous graphs under certain conditions. Our work carefully characterizes these conditions, and provides supporting theoretical understanding and empirical observations. Finally, we examine existing heterophilous graphs benchmarks and reconcile how the GCN (under)performs on them based on this understanding.Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

Is Homophily a Necessity for Graph Neural Networks?

Ma¹,

Liu²,

Shah³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…election. This dataset comes from the 2016 presidential election, where nodes are U.S. counties, and edges connect counties with the strongest Facebook social ties [26]. Each node has county-level demographic features (e.g., median income) and social network user features (e.g., fraction of friends within 50 miles).…”

Section: Methodsmentioning

confidence: 99%

“…Besides MRFs, other GNN approaches have used training labels at inference in graph-based learning. These include diffusions on training labels for data augmentation [23] and post-processing techniques [24,25,26]. Similar in spirit, smoothing techniques model positive label correlations in neighboring nodes [27,28].…”

Section: Mrfs With Gnns a Few Approaches Combine Mrfs And Graph Neura...mentioning

confidence: 99%

Graph Belief Propagation Networks

Jia,

Baykal,

Potluru

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

With the wide-spread availability of complex relational data, semi-supervised node classification in graphs has become a central machine learning problem. Graph neural networks are a recent class of easy-to-train and accurate methods for this problem that map the features in the neighborhood of a node to its label, but they ignore label correlation during inference and their predictions are difficult to interpret. On the other hand, collective classification is a traditional approach based on interpretable graphical models that explicitly model label correlations. Here, we introduce a model that combines the advantages of these two approaches, where we compute the marginal probabilities in a conditional random field, similar to collective classification, and the potentials in the random field are learned through end-to-end training, akin to graph neural networks. In our model, potentials on each node only depend on that node's features, and edge potentials are learned via a coupling matrix. This structure enables simple training with interpretable parameters, scales to large networks, naturally incorporates training labels at inference, and is often more accurate than related approaches. Our approach can be viewed as either an interpretable message-passing graph neural network or a collective classification method with higher capacity and modernized training.Preprint. Under review.

show abstract

“…In contrast, diffusion-like methods work precisely because of homophily and are typically fast. In the simple case of graphs, combining these two ideas has led to several recent advances [19,15,16].…”

Section: Introductionmentioning

confidence: 99%

A nonlinear diffusion method for semi-supervised learning on hypergraphs

Tudisco¹,

Prokopchik²,

Benson³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Hypergraphs are a common model for multiway relationships in data, and hypergraph semisupervised learning is the problem of assigning labels to all nodes in a hypergraph, given labels on just a few nodes. Diffusions and label spreading are classical techniques for semi-supervised learning in the graph setting, and there are some standard ways to extend them to hypergraphs. However, these methods are linear models, and do not offer an obvious way of incorporating node features for making predictions. Here, we develop a nonlinear diffusion process on hypergraphs that spreads both features and labels following the hypergraph structure, which can be interpreted as a hypergraph equilibrium network. Even though the process is nonlinear, we show global convergence to a unique limiting point for a broad class of nonlinearities, which is the global optimum of a interpretable, regularized semi-supervised learning loss function. The limiting point serves as a node embedding from which we make predictions with a linear model. Our approach is much more accurate than several hypergraph neural networks, and also takes less time to train.

show abstract

A Unifying Generative Model for Graph Learning Algorithms: Label Propagation, Graph Convolutions, and Combinations

Cited by 10 publications

References 35 publications

Is Homophily a Necessity for Graph Neural Networks?

Is Homophily a Necessity for Graph Neural Networks?

Graph Belief Propagation Networks

A nonlinear diffusion method for semi-supervised learning on hypergraphs

Contact Info

Product

Resources

About