Scalable distributed subgraph enumeration

Abstract

Subgraph enumeration aims to find all the subgraphs of a large data graph that are isomorphic to a given pattern graph. As the subgraph isomorphism operation is computationally intensive, researchers have recently focused on solving this problem in distributed environments, such as MapReduce and Pregel. Among them, the state-of-the-art algorithm, Twin TwigJoin, is proven to be instance optimal based on a left-deep join framework. However, it is still not scalable to large graphs because of the constraints in the left-deep join framework and that each decomposed component (join unit) must be a star. In this paper, we propose SEED - a scalable sub-graph enumeration approach in the distributed environment. Compared to Twin TwigJoin, SEED returns optimal solution in a generalized join framework without the constraints in Twin TwigJoin. We use both star and clique as the join units, and design an effective distributed graph storage mechanism to support such an extension. We develop a comprehensive cost model, that estimates the number of matches of any given pattern graph by considering power-law degree distribution in the data graph. We then generalize the left-deep join framework and develop a dynamic-programming algorithm to compute an optimal bushy join plan. We also consider overlaps among the join units. Finally, we propose clique compression to further improve the algorithm by reducing the number of the intermediate results. Extensive performance studies are conducted on several real graphs, one containing billions of edges. The results demonstrate that our algorithm outperforms all other state-of-the-art algorithms by more than one order of magnitude.

Fast and robust distributed subgraph enumeration

“…If E is still true after the local verification, we add (u, v) to f (Line 13). Then we create a new trie node N for v with N as its parentN (Line 14,15). After that, if f grows to an EC of Pi , then for each undetermined edge e of f (both end vertices are not in the local machine), we add N to I[e] (Line 17, 18).…”

“…Since join-based methods need to group the intermediate results based on keys so as to join them together, the performance was significantly dragged down when dealing with sparse graphs compared with RADS and PSgL. It is worth noting that PSgL was verified slower than TwinTwig and SEED in [13] [15]. This may be because the datasets used in TwinTwig and SEED are much denser than RoadNet, hence a huge number of embeddings will be generated.…”

“…It is observed in previous work [15,18] that when the data graph is large, the number of intermediate results can be huge, making the network communication cost a bottleneck and causing memory crash. On the other hand, systems that rely on replication of large parts of the data graph or heavy indexes are impractical for large data graphs and lowend computer clusters.…”

Distributed evaluation of subgraph queries using worst-case optimal low-memory dataflows

“…In doing so, we also achieve two things. First, we compare our work to the recent SEED [36] work, which develops efficient optimizations for evaluating undirected subgraph queries in the distributed setting. Second, by implementing one of the optimizations, we demonstrate that our approach can take as input general relations instead of the binary edge(ai, aj) relations we used so far.…”

Hybrid algorithms for subgraph pattern queries in graph databases

“…Below we refer to some interesting representative examples. Methods, such as TwinTwig [22], sTwig [6] and SEED [23] deal with a single, very large graph, stored in a distributed infrastructure, and rely on parallel computing algorithms and infrastructures to perform the sub-iso testing. Methods, like iGQ [24] and GraphCache [25], employ caching on top of any proposed FTV method to improve performance and study the architecture, system and algorithms for a graph cache for subgraph queries for FTV and SI methods.…”