Parallel subgraph listing in a large-scale graph

Shao, Yingxia; Cui, Bin; Chen, Lei; Ma, Lin; Yao, Junjie; Xu, Nuo

doi:10.1145/2588555.2588557

Cited by 110 publications

(59 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…PSgL [21] is based on Pregel [17]. It maps the query vertices one at a time following breath-first traversal, so that partial matches are expanded repeatedly until the final results are obtained.…”

Section: Related Workmentioning

confidence: 99%

Fast and robust distributed subgraph enumeration

et al. 2019

View full text Add to dashboard Cite

We study the classic subgraph enumeration problem under distributed settings. Existing solutions either suffer from severe memory crisis or rely on large indexes, which makes them impractical for very large graphs. Most of them follow a synchronous model where the performance is often bottlenecked by the machine with the worst performance. Motivated by this, in this paper, we propose RADS, a Robust Asynchronous Distributed Subgraph enumeration system. RADS first identifies results that can be found using singlemachine algorithms. This strategy not only improves the overall performance but also reduces network communication and memory cost. Moreover, RADS employs a novel region-grouped multi-round expand verify & filter framework which does not need to shuffle and exchange the intermediate results, nor does it need to replicate a large part of the data graph in each machine. This feature not only reduces network communication cost and memory usage, but also allows us to adopt simple strategies for memory control and load balancing, making it more robust. Several heuristics are also used in RADS to further improve the performance. Our experiments verified the superiority of RADS to state-of-the-art subgraph enumeration approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

Fast and robust distributed subgraph enumeration

et al. 2019

View full text Add to dashboard Cite

show abstract

“…The BFS-style algorithms enumerate matching results of join units first and assemble them via one or more rounds of joining to get the matching results for the whole pattern graph. The algorithms shuffle the partial matching results ( [17] 3.1E6 1.2E8 6.3E8 3.2E9 6.7E10 uk-2002 (uk) [18] 1.8E7 2.6E8 4.4E9 1.6E11 2.7E12 FriendSter (fs) [17] 6.5E7 1.8E9 4.2E9 9.0E9 1.8E11 varieties of join units (Edge [19], Star [8] [20] , TwinTwig [15] [ 16], Clique [8], Crystal [9]) and join frameworks (Left-deep join [15], [16], Bushy join [8], Hash-assembly [9], Generic join [19]) to reduce intermediate results.…”

Section: Motivationmentioning

confidence: 99%

BENU: Distributed Subgraph Enumeration with Backtracking-Based Framework

Wang

et al. 2019

2019 IEEE 35th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Given a small pattern graph and a large data graph, the task of subgraph enumeration is to find all subgraphs of the data graph that are isomorphic to the pattern graph. When the data graph is dynamic, the task of continuous subgraph enumeration is to detect the changes in the matching results caused by the edge updates at each time step. The two tasks are fundamental in many graph analysis applications. The state-of-the-art distributed methods solve them via distributed multi-way join. However, they are inefficient in communication since they have to shuffle partial matching results during the join. The partial matching results may be much larger than the data graph itself. To overcome the drawback, we develop the BENU framework for distributed subgraph enumeration. Given a data graph, BENU generates a group of local search tasks that follow a backtracking-based execution plan to enumerate subgraphs of the pattern graph. BENU executes the tasks in parallel. The tasks query the data graph stored in a distributed database on demand, avoid shuffling partial matching results. To handle dynamic data graphs, we propose the concept of incremental pattern graphs. We solve the continuous subgraph enumeration via enumerating incremental pattern graphs in the data graph snapshots at each time step. We extend BENU into S-BENU to enumerate them efficiently. We develop implementations for BENU and S-BENU with two optimization techniques. The extensive experiments show that BENU and S-BENU are scalable. They outperform the state-of-the-art by up to one and two orders of magnitude, respectively.Index Terms-backtracking-based framework, continuous subgraph matching, distributed graph querying, subgraph isomorphism, subgraph matching. !

show abstract

“…PSgL [50]: PSgL is a subgraph enumeration system that is built on top of Giraph [21]. PSgL picks an order of the vertices (i.e., attributes), say a1, ..., am in Q, called a traversal order.…”

Section: Related Workmentioning

confidence: 99%

Distributed evaluation of subgraph queries using worst-case optimal low-memory dataflows

et al. 2018

View full text Add to dashboard Cite

We study the problem of finding and monitoring fixed-size subgraphs in a continually changing large-scale graph. We present the first approach that (i) performs worst-case optimal computation and communication, (ii) maintains a total memory footprint linear in the number of input edges, and (iii) scales down per-worker computation, communication, and memory requirements linearly as the number of workers increases, even on adversarially skewed inputs.Our approach is based on worst-case optimal join algorithms, recast as a data-parallel dataflow computation. We describe the general algorithm and modifications that make it robust to skewed data, prove theoretical bounds on its resource requirements in the massively parallel computing model, and implement and evaluate it on graphs containing as many as 64 billion edges. The underlying algorithm and ideas generalize from finding and monitoring subgraphs to the more general problem of computing and maintaining relational equi-joins over dynamic relations. Edge-at-a-time ApproachesPerhaps the most common approach to finding instances of a query subgraph is to treat it as a relational query, and to execute a sequence of binary joins to determine the result. For example,

show abstract

Parallel subgraph listing in a large-scale graph

Cited by 110 publications

References 23 publications

Fast and robust distributed subgraph enumeration

Fast and robust distributed subgraph enumeration

BENU: Distributed Subgraph Enumeration with Backtracking-Based Framework

Distributed evaluation of subgraph queries using worst-case optimal low-memory dataflows

Contact Info

Product

Resources

About