Bisimulation Reduction of Big Graphs on MapReduce

Luo, Yongming; Lange, Yannick de; Fletcher, George H. L.; Bra, Paul De; Hidders, Jan; Wu, Yuqing

doi:10.1007/978-3-642-39467-6_18

Cited by 16 publications

(23 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Independently and parallel to our work, [18] have proposed a MapReduce-based implementation of bisimulation. While they seem to be mostly interested in a clever handling of skew data, we aim at reducing the overall number of MapReduce tasks required for computing a bisimulation reduction, i.e.…”

Section: Related Workmentioning

confidence: 99%

“…To give a rough comparison of the evaluation results of the different implementations, we first have to comment on the Hadoop clusters used. While our cluster consists of 10 nodes each having a 1.9 GHz 6-Core CPU with 32 GB RAM, the cluster described in [18] has a size of 72 nodes each being equipped with a 2.0 GHz 8-Core CPU with 64 GB RAM. Our DBPedia dataset consists of 61.06M nodes and 198.09M triples, while theirs consists of only of 38.62M nodes and 115.3M triples.…”

Section: Related Workmentioning

confidence: 99%

“…Experiments based on synthetic benchmark data and real data (DBPedia) exhibit a reduction of more than 90% of the size of the RDF graph in terms of the number of nodes to the number of blocks in the resulting bisimulation partition. Independently and parallel to our work, [18] also have proposed a MapReduce-based implementation of bisimulation. However, as we will detail later in this paper, our and their work cannot easily be compared because of the different emphasis.…”

Section: Introductionmentioning

confidence: 98%

See 2 more Smart Citations

Large-scale bisimulation of RDF graphs

Schätzle

Neu

Lausen

et al. 2013

Proceedings of the Fifth Workshop on Semantic Web Information Management

View full text Add to dashboard Cite

RDF datasets with billions of triples are no longer unusual and continue to grow constantly (e.g. LOD cloud) driven by the inherent flexibility of RDF that allows to represent very diverse datasets, ranging from highly structured to unstructured data. Because of their size, understanding and processing RDF graphs is often a difficult task and methods to reduce the size while keeping as much of its structural information become attractive. In this paper we study bisimulation as a means to reduce the size of RDF graphs according to structural equivalence. We study two bisimulation algorithms, one for sequential execution using SQL and one for distributed execution using MapReduce. We demonstrate that the MapReduce-based implementation scales linearly with the number of the RDF triples, allowing to compute the bisimulation of very large RDF graphs within a time which is by far not possible for the sequential version. Experiments based on synthetic benchmark data and real data (DBPedia) exhibit a reduction of more than 90% of the size of the RDF graph in terms of the number of nodes to the number of blocks in the resulting bisimulation partition.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 98%

See 1 more Smart Citation

Large-scale bisimulation of RDF graphs

Schätzle

Neu

Lausen

et al. 2013

Proceedings of the Fifth Workshop on Semantic Web Information Management

View full text Add to dashboard Cite

show abstract

“…However, to our knowledge there is no known effective solution for computing bisimulation and k-bisimulation partitions on arbitrary graph structures in external memory. Such an algorithm would not only enable us to process big graphs on single machines, but also provide an essential step for parallel and distributed solutions (e.g., MapReduce [20]) to further scale their performance on real graphs. As noted in paper [20] and many other researches (e.g., [19]), in many cases, single machine external memory algorithms are more competitive than distributed algorithms due to their lack of communication overhead and their effective use of available infrastructure.…”

Section: State Of the Artmentioning

confidence: 99%

External memory K-bisimulation reduction of big graphs

Luo

Fletcher

Hidders

et al. 2013

Proceedings of the 22nd ACM International Conference on Information &Amp; Knowledge Management

Self Cite

View full text Add to dashboard Cite

In this paper, we present, to our knowledge, the first known I/O efficient solutions for computing the k-bisimulation partition of a massive directed graph, and performing maintenance of such a partition upon updates to the underlying graph. Ubiquitous in the theory and application of graph data, bisimulation is a robust notion of node equivalence which intuitively groups together nodes in a graph which share fundamental structural features. kbisimulation is the standard variant of bisimulation where the topological features of nodes are only considered within a local neighborhood of radius k 0.The I/O cost of our partition construction algorithm is bounded by O(k · sort(|Et|) + k · scan(|Nt|) + sort(|Nt|)), while our maintenance algorithms are bounded by O(k · sort(|Et|) + k · sort(|Nt|)). The space complexity bounds are O(|Nt| + |Et|) and O(k · |Nt| + k · |Et|), resp. Here, |Et| and |Nt| are the number of disk pages occupied by the input graph's edge set and node set, resp., and sort(n) and scan(n) are the cost of sorting and scanning, resp., a file occupying n pages in external memory. Empirical analysis on a variety of massive real-world and synthetic graph datasets shows that our algorithms perform efficiently in practice, scaling gracefully as graphs grow in size.

show abstract

“…Algorithms targeting various constraints and computational models, such as main-memory algorithms [1], I/O efficient algorithms [16,22] and distributed solutions [6,21], have been developed to efficiently compute bisimulation partitions of massive graphs. For example, the state-of-the-art MapReduce-based algorithm [21] can compute a "k-localized" variant of bisimulation, discussed below, on a social graph with 1.4 billion edges in a few hours, for k = 10.…”

Section: Introductionmentioning

confidence: 99%

Regularities and dynamics in bisimulation reductions of big graphs

Luo

Fletcher

Hidders

et al. 2013

First International Workshop on Graph Data Management Experiences and Systems

Self Cite

View full text Add to dashboard Cite

Bisimulation is a basic graph reduction operation, which plays a key role in a wide range of graph analytical applications. While there are many algorithms dedicated to computing bisimulation results, to our knowledge, little work has been done to analyze the results themselves. Since data properties such as skew can greatly influence the performances of data-intensive tasks, the lack of such insight leads to inefficient algorithm and system design.In this paper we take a close look into various aspects of bisimulation results on big graphs, from both real-world scenarios and synthetic graph generators, with graph size varying from 1 million to 1 billion edges. We make the following observations: (1) A certain degree of regularity exists in real-world graphs' bisimulation results. Specifically, power-law distributions appear in many of the results' properties.(2) Synthetic graphs fail to fulfill one or more of these regularities that are revealed in the real-world graphs.(3) By examining a growing social network graph (FlickrGrow), we see that the corresponding bisimulation partition relation graph grows as well, but the growth is stable with respect to the original graph.

show abstract

Bisimulation Reduction of Big Graphs on MapReduce

Cited by 16 publications

References 17 publications

Large-scale bisimulation of RDF graphs

Large-scale bisimulation of RDF graphs

External memory K-bisimulation reduction of big graphs

Regularities and dynamics in bisimulation reductions of big graphs

Contact Info

Product

Resources

About