Learning Edge Representations via Low-Rank Asymmetric Projections

Abstract

We propose a new method for embedding graphs while preserving directed edge information. Learning such continuous-space vector representations (or embeddings) of nodes in a graph is an important first step for using network information (from social networks, user-item graphs, knowledge bases, etc.) in many machine learning tasks. Unlike previous work, we (1) explicitly model an edge as a function of node embeddings, and we (2) propose a novel objective, the "graph likelihood", which contrasts information from sampled random walks with non-existent edges. Individually, both of these contributions improve the learned representations, especially when there are memory constraints on the total size of the embeddings. When combined, our contributions enable us to significantly improve the state-of-the-art by learning more concise representations that better preserve the graph structure. We evaluate our method on a variety of link-prediction task including social networks, collaboration networks, and protein interactions, showing that our proposed method learn representations with error reductions of up to 76% and 55%, on directed and undirected graphs. In addition, we show that the representations learned by our method are quite space efficient, producing embeddings which have higher structure-preserving accuracy but are 10 times smaller

Is a Single Embedding Enough? Learning Node Representations that Capture Multiple Social Contexts

“…For each experiment, we report the AUC-ROC in a link prediction task performed using an ablation test described in Section 3.2. For consistency of comparison, we use the experimental settings (datasets and training/testing splits) of [2] for the baselines. Hence, the baselinesâĂŹ numbers are the same of [2] and are reported for completeness.…”

“…3 During inference with the baselines, we use the embedding of a pair of node u and v to rank the likelihood of the link u, v formed by employing a scoring function that takes in the input the embeddings of the two nodes. To do so, for consistency with previous work, we used the same methodology of [2], which we summarize here. Let Y u and Y v be, respectively, the embeddings of u and v. The edge scoring function is defined as follows: for EigenMaps, it is −||Y u − Y v ||; for node2vec, we use the off-shelve binary classification LogisticRegression algorithm of sklearn to lean a model over the Hadamard product of the embeddings of the two nodes; for DNGR, we use the bottleneck layer values as the embeddings and the dot product as similarity; for Asymmetric, we use the dot product; and for M-NMF, similarly to node2vec, we train a model on the Hadamard product of the embeddings.…”

“…First, we are interested in developing new models that can capture the variation of social behaviors that are expressed in realworld social networks (such as membership in multiple communities). Several recent works suggest that link prediction (or network reconstruction) [2,50] is the best way to analyze an unsupervised network embedding's performance, as it is a primary task (unlike node classification -a secondary task that involves a labeling process that may be uncorrelated with the graph itself). Second, there are several important industrial applications of link prediction on real networks (e.g.…”

“…The Link Prediction task follows the methodology introduced in [2,25], which we briefly detail here. First, the input graph is split into two edge sets, E train and E test , of equal size.…”

“…In a nutshell, an embedding algorithm learns a latent vector representation that maps each vertex v in the graph G to a single d dimensional vector. This area has found strong applications, as the embedding representation of nodes leads to improved results in data mining and machine learning tasks, such as node classification [37], user profiling [39], ranking [26], and link prediction [2,25]. In virtually all cases, the crucial assumption of the embedding methods developed so far is that a single embedding vector has to be learned for each node in the graph.…”