A Recurrent Graph Neural Network for Multi-relational Data

Abstract

The era of "data deluge" has sparked the interest in graph-based learning methods in a number of disciplines such as sociology, biology, neuroscience, or engineering. In this paper, we introduce a graph recurrent neural network (GRNN) for scalable semisupervised learning from multi-relational data. Key aspects of the novel GRNN architecture are the use of multi-relational graphs, the dynamic adaptation to the different relations via learnable weights, and the consideration of graph-based regularizers to promote smoothness and alleviate over-parametrization. Our ultimate goal is to design a powerful learning architecture able to: discover complex and highly non-linear data associations, combine (and select) multiple types of relations, and scale gracefully with respect to the size of the graph. Numerical tests with real datasets corroborate the design goals and illustrate the performance gains relative to competing alternatives.

Physics-Aware Neural Networks for Distribution System State Estimation

“…The graph convolutional NN (GCNN) [21], [22] learning approaches are designed to process data defined over graphs. These models also lead to sparsification of the weight matrices connecting the hidden layers in the neural network.…”

“…2b are constrained to be scaled versions of each other, then the proposed method becomes equivalent to the GCNN in [21]. On the other hand, if the vertically aligned blocks in the weight matrices of PAWNN are chosen to be scaled versions of a fixed matrix, then the learning model of [22] emerges as a special case. For example, for the Fig.…”

“…2, if we constrain the blocks (3, 4), (5,4), and (6, 4) of the weights matrix in Fig. 2b to be scaled versions of the block (4,4), and similarly for all other vertically aligned blocks, then the model reduces to the graph NN proposed in [22].…”

Gated Graph Recurrent Neural Networks

“…Related work on GRNNs [22]- [25] considers only regression problems and is thus limited to outputting sequences of graph signals, with [22]- [24] targeting traffic forecasting specifically. Other somewhat related works include the gated graph sequence neural networks [26] and the recurrent formulation in [27]. The architecture in [26] learns sequential representations from graphs, and not from graph signals or processes; this is a fundamental difference, since in learning from graphs the graph is seen as data, while in learning from graph signals the graph is given (i.e.…”

“…a hyperparameter of the learning architecture). The work in [27] uses recurrence as a means of re-introducing the input at arXiv:2002.01038v1 [eess.SP] 3 Feb 2020 every layer to capture multiple types of diffusion, but does not consider data consisting of temporal sequences. While time gating has been discussed in [22]- [26], the novel spatial gating strategies that we put forward leverage the graph's node and edge structures to control how long range spatial dependencies on the graph are encoded.…”