VENUS: A System for Streamlined Graph Computation on a Single PC

Liu, Qin; Cheng, Jian‐Wen; Li, Zhenguo; Lui, John C. S.

doi:10.1109/tkde.2015.2502938

Cited by 3 publications

(4 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kim et al [36] present a parallel disk-based triangle enumeration system OPT on a single machine by exploiting the features of solid-state drive (SSD) and multi-core CPU parallelism. General-purpose disk-based graph computing systems (e.g., GraphChi [37], X-Stream [38], TurboGraph [39], VENUS [40], and NXgraph [41]) also provide an implementation for counting triangles. These algorithms and systems are not customized for dealing with graph streams because they require that the entire graph of interest is given in advance.…”

Section: A Counting Triangles On Just a Machinementioning

confidence: 99%

REPT: A Streaming Algorithm of Approximating Global and Local Triangle Counts in Parallel

Wang

Jia

et al. 2019

2019 IEEE 35th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Recently, considerable efforts have been devoted to approximately computing the global and local (i.e., incident to each node) triangle counts of a large graph stream represented as a sequence of edges. Existing approximate triangle counting algorithms rely on sampling techniques to reduce the computational cost. However, their estimation errors are significantly determined by the covariance between sampled triangles. Moreover, little attention has been paid to developing parallel one-pass streaming algorithms that can be used to fast and approximately count triangles on a multi-core machine or a cluster of machines. To solve these problems, we develop a novel parallel method REPT to significantly reduce the covariance (even completely eliminate the covariance for some cases) between sampled triangles. We theoretically prove that REPT is more accurate than parallelizing existing triangle count estimation algorithms in a direct manner. In addition, we also conduct extensive experiments on a variety of real-world graphs, and the results demonstrate that our method REPT is several times more accurate than state-of-the-art methods. c c i=1τ (i) , where c is the number of processors used for estimating the global triangle count τ . In this case, the arXiv:1811.09136v2 [cs.DS]

show abstract

Section: A Counting Triangles On Just a Machinementioning

confidence: 99%

REPT: A Streaming Algorithm of Approximating Global and Local Triangle Counts in Parallel

Wang

Jia

et al. 2019

2019 IEEE 35th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

show abstract

“…To further accelerate the preprocessing, we reimplement DrunkardMob on our system, VENUS [30] (a disk-based graph computation system in C++) and extend it in a distributed environment using MPI [34]. PowerWalk distributes the simulation of random walks to a set of workers.…”

Section: Preprocessingmentioning

confidence: 99%

“…The first system is designed for the MapReduce model, which aims to optimize the I/O efficiency [4]. The other system, called DrunkardMob [25], is designed for single-machine disk-based graph engines like GraphChi [26] and VENUS [30]. Although DrunkardMob works on a single machine, since it stores the graph on disk, it can still handle graphs larger than the main memory.…”

Section: Related Workmentioning

confidence: 99%

“…Furthermore, we propose the VERD algorithm which can execute PPR queries online based on the PPR index obtained from the MCFP algorithm. Our VERD algorithm could also be realized on disk-based graph engines like GraphChi [26] and VENUS [30] that support vertex-centric programming. Since at each iteration, a disk-based graph engine will scan the entire graph on disk, the I/O cost is at least…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

PowerWalk

Liu

Lui

et al. 2016

Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

Self Cite

View full text Add to dashboard Cite

Most methods for Personalized PageRank (PPR) precompute and store all accurate PPR vectors, and at query time, return the ones of interest directly. However, the storage and computation of all accurate PPR vectors can be prohibitive for large graphs, especially in caching them in memory for real-time online querying. In this paper, we propose a distributed framework that strikes a better balance between offline indexing and online querying. The offline indexing attains a fingerprint of the PPR vector of each vertex by performing billions of "short" random walks in parallel across a cluster of machines. We prove that our indexing method has an exponential convergence, achieving the same precision with previous methods using a much smaller number of random walks. At query time, the new PPR vector is composed by a linear combination of related fingerprints, in a highly efficient vertex-centric decomposition manner. Interestingly, the resulting PPR vector is much more accurate than its offline counterpart because it actually uses more random walks in its estimation. More importantly, we show that such decomposition for a batch of queries can be very efficiently processed using a shared decomposition. Our implementation, PowerWalk, takes advantage of advanced distributed graph engines and it outperforms the state-of-the-art algorithms by orders of magnitude. Particularly, it responses to tens of thousands of queries on graphs with billions of edges in just a few seconds.

show abstract