Temporal Graph Offset Reconstruction: Towards Temporally Robust Graph Representation Learning

Abstract

Graphs are a commonly used construct for representing relationships between elements in complex high dimensional datasets. Many real-world phenomenon are dynamic in nature, meaning that any graph used to represent them is inherently temporal. However, many of the machine learning models designed to capture knowledge about the structure of these graphs ignore this rich temporal information when creating representations of the graph. This results in models which do not perform well when used to make predictions about the future state of the graph -especially when the delta between time stamps is not small. In this work, we explore a novel training procedure and an associated unsupervised model which creates graph representations optimised to predict the future state of the graph. We make use of graph convolutional neural networks to encode the graph into a latent representation, which we then use to train our temporal offset reconstruction method, inspired by auto-encoders, to predict a later time point -multiple time steps into the future. Using our method, we demonstrate superior performance for the task of future link prediction compared with none-temporal stateof-the-art baselines. We show our approach to be capable of outperforming non-temporal baselines by 38% on a real world dataset.

Temporal network embedding framework with causal anonymous walks representations

“…Static models are used on the graph snapshots to make predictions for the next one (DNE ( Du et al, 2018 ), TO-GAE ( Bonner et al, 2018 ), DynGEM ( Goyal et al, 2018 )). Another way of implementing this methodology, called TI-GCN (Time Interval Graph Convolutional Networks) via residual architectures.…”

A Survey on Embedding Dynamic Graphs

“…These approaches have been also explored by several dynamic graph embedding methods [71,81]. Even further, instead of using RNNs to explore the characteristics of the network over time, many succeeding models also employ convolutional networks (for instance, 1D CNNs) to leverage temporal dependencies [82,53,83,84].…”

“…Variational graph autoencoder (VGAE) [87] and Graph-GAN [88] employ these approaches to static graphs. These generative models are used in dynamic graphs to learn these data distributions over time [82,31,89,90,91,84,44].…”

“…The output is the reconstructed graph at a future time. Bonner et al [82] regarded this approach as the temporal graph offset reconstruction problem, i.e. creating temporal graph embeddings that recreate a future timestamp of the graph.…”

“…Algorithm Deep Learning Model TO-GAE [82] GCNs over time dyngraph2vecAE [27] Fully-Connected (FC) Autoencoders dyngraph2vecRNN [27] Sparsely Connected LSTMs dyngraph2vecAERNN [27] FC Encoder, LSTMs and FC Decoder E-LSTM-D [74] FC Encoder, LSTMs and FC Decoder GC-LSTM [71] Graph Convolutions and LSTMs EvolveGCN-H [47] GCNs and GRUs EvolveGCN-O [47] GCNs and LSTMs T-GCN [83] GCNs and GRUs…”