Graph Belief Propagation Networks

Jia, Junteng; Baykal, Cenk; Potluru, Vamsi K.; Benson, Austin R.

doi:10.48550/arxiv.2106.03033

Cited by 3 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some recent studies combine methods from collective classification with neural networks to ensure a better end-to-end learning, e.g. [26]. However, all the above-mentioned algorithms only make use of features of first-hop neighbors and thus rely on a propagation step to make use of higher-hop nodes' information in an iterative manner.…”

Section: Coramentioning

confidence: 99%

“…It is worth pointing out that a semi-supervised estimation approach may also be possible but this would require an iterative approach that cycles between parameter estimation and node classification. It is also worth noting that the estimation step plays the role of the learning phase in GNN-based classifiers or in the classifiers developed in [26].…”

Section: B Parameters' Estimationmentioning

confidence: 99%

“…In Table 6, we compare our approaches GABγ and GAB⋆ up to five hops (in GAB⋆, we approximate the unknown distributions by two-layer NNs which are learnt during the training phase) with the methods presented and analyzed in [26]. We copy-paste the values given in [26] where the best number of layers and the best version of each approach have been selected. We select the best version of GAB in the sense that we optimized the hyperparameters γ by allowing at most the fifth-order case.…”

Section: B Real Datasetsmentioning

confidence: 99%

See 2 more Smart Citations

Graph-Assisted Bayesian Node Classifiers

et al. 2023

View full text Add to dashboard Cite

Many datasets can be represented by attributed graphs on which classification methods may be of interest. The problem of node classification has attracted the attention of scholars due to its wide range of applications. The problem consists of predicting nodes' labels based on their intrinsic features, features of their neighboring nodes and the graph structure. Graph Neural Networks (GNN) have been widely used to tackle this task. Thanks to the graph structure and the node features, they are able to propagate information over the graph and aggregate it to improve the classification performance. Their performance is however sensitive to the graph topology, especially its degree of impurity, a measure of the proportion of connected nodes belonging to different classes. Here, we propose a new Graph-Assisted Bayesian (GAB) classifier, which is designed for the problem of node classification. By using the Bayesian theorem, GAB takes into consideration the degree of impurity of the graph when classifying the nodes. We show that the proposed classifier is less sensitive to graph impurity, and less complex than GNN-based classifiers. INDEX TERMSNode classification, Attributed graphs, Degree of Impurity, Bayesian framework I. INTRODUCTION

show abstract

Section: Coramentioning

confidence: 99%

Section: B Parameters' Estimationmentioning

confidence: 99%

Section: B Real Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Graph-Assisted Bayesian Node Classifiers

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Our results imply that ReLU is the de facto optimal non-linearity instead of attention and may at most marginally outperform the linear model when with low-quality node attributes. Some previous works also use Bayesian inference to inspire GNN architectures [73][74][75][76][77][78][79][80], while these works focus on empirical evaluation instead of theoretical analysis.…”

Section: More Related Workmentioning

confidence: 99%

Understanding Non-linearity in Graph Neural Networks from the Bayesian-Inference Perspective

Wei¹,

Yin²,

Jia³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Graph neural networks (GNNs) have shown superiority in many prediction tasks over graphs due to their impressive capability of capturing nonlinear relations in graph-structured data. However, for node classification tasks, often, only marginal improvement of GNNs over their linear counterparts has been observed. Previous works provide very few understandings of this phenomenon. In this work, we resort to Bayesian learning to deeply investigate the functions of non-linearity in GNNs for node classification tasks. Given a graph generated from the statistical model CSBM, we observe that the max-a-posterior estimation of a node label given its own and neighbors' attributes consists of two types of non-linearity, a possibly nonlinear transformation of node attributes and a ReLU-activated feature aggregation from neighbors. The latter surprisingly matches the type of non-linearity used in many GNN models. By further imposing Gaussian assumption on node attributes, we prove that the superiority of those ReLU activations is only significant when the node attributes are far more informative than the graph structure, which nicely matches many previous empirical observations. A similar argument can be achieved when there is a distribution shift of node attributes between the training and testing datasets. Finally, we verify our theory on both synthetic and real-world networks.Preprint. Under review.

show abstract

“…Combining NNs with BP is not new (e.g. [Kuck et al, 2020, Jia et al, 2021); the innovation is that it is done in a fully general way and accomplished with little extra work in a common framework.…”

Section: Introductionmentioning

confidence: 99%

Unifying AI Algorithms with Probabilistic Programming using Implicitly Defined Representations

Pfeffer,

Harradon,

Campolongo

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce Scruff, a new framework for developing AI systems using probabilistic programming. Scruff enables a variety of representations to be included, such as code with stochastic choices, neural networks, differential equations, and constraint systems. These representations are defined implicitly using a set of standardized operations that can be performed on them. General-purpose algorithms are then implemented using these operations, enabling generalization across different representations. Zero, one, or more operation implementations can be provided for any given representation, giving algorithms the flexibility to use the most appropriate available implementations for their purposes and enabling representations to be used in ways that suit their capabilities. In this paper, we explain the general approach of implicitly defined representations and provide a variety of examples of representations at varying degrees of abstraction. We also show how a relatively small set of operations can serve to unify a variety of AI algorithms. Finally, we discuss how algorithms can use policies to choose which operation implementations to use during execution.

show abstract

Graph Belief Propagation Networks

Cited by 3 publications

References 24 publications

Graph-Assisted Bayesian Node Classifiers

Graph-Assisted Bayesian Node Classifiers

Understanding Non-linearity in Graph Neural Networks from the Bayesian-Inference Perspective

Unifying AI Algorithms with Probabilistic Programming using Implicitly Defined Representations

Contact Info

Product

Resources

About