MILE: A Multi-Level Framework for Scalable Graph Embedding

Abstract

Recently there has been a surge of interest in designing graph embedding methods. Few, if any, can scale to a large-sized graph with millions of nodes due to both computational complexity and memory requirements. In this paper, we relax this limitation by introducing the MultI-Level Embedding (MILE) framework -a generic methodology allowing contemporary graph embedding methods to scale to large graphs. MILE repeatedly coarsens the graph into smaller ones using a hybrid matching technique to maintain the backbone structure of the graph. It then applies existing embedding methods on the coarsest graph and refines the embeddings to the original graph through a novel graph convolution neural network that it learns. The proposed MILE framework is agnostic to the underlying graph embedding techniques and can be applied to many existing graph embedding methods without modifying them. We employ our framework on several popular graph embedding techniques and conduct embedding for real-world graphs. Experimental results on five large-scale datasets demonstrate that MILE significantly boosts the speed (order of magnitude) of graph embedding while also often generating embeddings of better quality for the task of node classification. MILE can comfortably scale to a graph with 9 million nodes and 40 million edges, on which existing methods run out of memory or take too long to compute on a modern workstation.

Scaling Up Graph Neural Networks Via Graph Coarsening

“…Graph reduction with spectral approximation guarantees are studied in [18,23,27]. Recently, graph coarsening has been applied to speedup graph embedding algorithms [12,16,24]. As far as we are aware, this is the first work applying graph coarsening to speedup the training of GNNs in the semi-supervised setting.…”

REFINE: Random RangE FInder for Network Embedding

“…REFINE also outperforms the initial RBQR (Algorithm 1) by a large margin. We also conduct the comparison with fast network embedding methods without matrix factorization [1,13,3]. As shown in Table 4, our method outperforms other methods significantly.…”

COSINE: Compressive Network Embedding on Large-scale Information Networks

“…Afterward, network embedding methods are applied to learn the representations of supernodes, and then with the learned representation as the initial value of the supernodes' constituent nodes, the embedding methods are run over finer-grained subgraphs again. Compared with HARP, MILE [34] implements embeddings refinement to learn better representations for nodes in finer-grained networks with lower computational cost and higher flexibility. While HARP and MILE still follow the setting of embedding lookup as previous work did, our framework manages to reduce the memory usage as well as improve the scalability.…”

“…HARP [33] and MILE [34] have used Graph Coarsening to find a smaller network which approximates the global structure of its input and learn coarse embeddings from the small network, which serve as good initializations for learning representation in the input network. Graph Coarsening coarsens a network without counting the number of origin nodes that belong to a coarsen group.…”

“…Yelp [34] contains 8, 981, 389 users and 39, 846, 890 social relations between them. The labels here represent the business categories on which the users have reviewed.…”