VENUS: Vertex-centric streamlined graph computation on a single PC

Cheng, Jian‐Wen; Liu, Qin; Li, Zhenguo; Wang, Fan; Lui, John C. S.; He, Cheng

doi:10.1109/icde.2015.7113362

Cited by 82 publications

(44 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In-Memory Out-of-Core (use HDD if not indicated) In-Memory Out-of-Core In-Memory Out-of-Core Approaches Ligra [7] Galois [8] GraphMat [9] Polymer [10] GraphChi [31] X-Stream [32] VENUS [33] GridGraph [34] GraphMP (Use SSD) FlashGraph [35] TurboGraph [36] Medusa [11] Gunrock [13] MapGraph [14] gGraph [15] (Use SSD) GTS [16] GGraph [15] Pregel-like: [20] [21] [22] [23] [24] GAS: [25] [26] [27] SpMV: [28] [29] (Use HDD) GraphD [37] Chaos [38] Pregelix [ GraphChi, the edge-centric scatter-gather (ESG) model of X-Stream, the vertex-centric streamlined processing (VSP) model of VENUS, and the dual sliding windows (DSW) model of GridGraph. These approaches try to exploit the sequential bandwidth of hard disks and to reduce the amount of disk accesses.…”

Section: Single Machine (Cpu) Single Machine (Gpu) Cluster Data Storagementioning

confidence: 99%

“…Out-of-core systems, which maintain just a small portion of vertices and/or edges in memory, provide cost-effective solutions for big graph analytics. Single-machine engines, such as GraphChi [31], X-Stream [32], VENUS [33] and GridGraph [34], break the input graph into a set of shards, each of which contains all required information to update its associated vertices. In many cases, an out-of-core graph engine processes all shards in an iteration, and usually uses three stages to execute a shard:…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

GraphMP: I/O-Efficient Big Graph Analytics on a Single Commodity Machine

Sun

Wen

Duong

et al. 2020

IEEE Trans. Big Data

View full text Add to dashboard Cite

Recent studies showed that single-machine graph processing systems can be as highly competitive as cluster-based approaches on large-scale problems. While several out-of-core graph processing systems and computation models have been proposed, the high disk I/O overhead could significantly reduce performance in many practical cases. In this paper, we propose GraphMP to tackle big graph analytics on a single machine. GraphMP achieves low disk I/O overhead with three techniques. First, we design a vertex-centric sliding window (VSW) computation model to avoid reading and writing vertices on disk. Second, we propose a selective scheduling method to skip loading and processing unnecessary edge shards on disk. Third, we use a compressed edge cache mechanism to fully utilize the available memory of a machine to reduce the amount of disk accesses for edges. Extensive evaluations have shown that GraphMP could outperform existing single-machine out-of-core systems such as GraphChi, X-Stream and GridGraph by up to 30, and can be as highly competitive as distributed graph engines like Pregel+, PowerGraph and Chaos.

show abstract

Section: Single Machine (Cpu) Single Machine (Gpu) Cluster Data Storagementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

GraphMP: I/O-Efficient Big Graph Analytics on a Single Commodity Machine

Sun

Wen

Duong

et al. 2020

IEEE Trans. Big Data

View full text Add to dashboard Cite

show abstract

“…A plethora of efforts using vertex-centric computing exist to perform bulk synchronous parallel graph computations. Some run only on one node, [7], [29], [15] while some support distributed computations, [14], [6], [26], [4], [20]. Some support only in-memory computations [21], [6] while others can utilize out-of-core [7], [20], [29], [15].…”

Section: Related Workmentioning

confidence: 99%

Streaming Temporal Graphs: Subgraph Matching

Goodman

Grunwald

2019

2019 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

We investigate solutions to subgraph matching within a temporal stream of data. We present a high-level language for describing temporal subgraphs of interest, the Streaming Analytics Language (SAL). SAL programs are translated into C++ code that is run in parallel on a cluster. We call this implementation of SAL the Streaming Analytics Machine (SAM). SAL programs are succinct, requiring about 20 times fewer lines of code than using the SAM library directly, or writing an implementation using Apache Flink. To benchmark SAM we calculate finding temporal triangles within streaming netflow data. Also, we compare SAM to an implementation written for Flink. We find that SAM is able to scale to 128 nodes or 2560 cores, while Apache Flink has max throughput with 32 nodes and degrades thereafter. Apache Flink has an advantage when triangles are rare, with max aggregate throughput for Flink at 32 nodes greater than the max achievable rate of SAM. In our experiments, when triangle occurrence was faster than five per second per node, SAM performed better. Both frameworks may miss results due to latencies in network communication. SAM consistently reported an average of 93.7% of expected results while Flink decreases from 83.7% to 52.1% as we increase to the maximum size of the cluster. Overall, SAM can obtain rates of 91.8 billion netflows per day.

show abstract

“…Instead, many graph computing engines such as Pregel [33] and PowerGraph [17] are being developed, which can be orders of magnitude more efficient than general-purpose dataflow systems [33,17,18]. A key feature of most graph engines is the adoption of a simple yet general vertexcentric programming model that can succinctly express a wide variety of graph algorithms [33,18,17,12]. Here, a graph algorithm is formulated into a user-defined vertex-program which can be executed on each vertex in parallel.…”

Section: Introductionmentioning

confidence: 99%

“…While the power iteration method is used in most graph engines to calculate the global PageRank [33,18,17,12], it is impractical for PPR computation, which would take O(N (N + M )) time for all PPR vectors (N and M are the numbers of vertices and edges respectively) [6]. Furthermore, most algorithms for PPR computation are designed for single-machine in-memory systems which cannot deal with large-scale problems [38,32,31].…”

Section: Introductionmentioning

confidence: 99%

PowerWalk

Liu

Lui

et al. 2016

Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

Self Cite

View full text Add to dashboard Cite

Most methods for Personalized PageRank (PPR) precompute and store all accurate PPR vectors, and at query time, return the ones of interest directly. However, the storage and computation of all accurate PPR vectors can be prohibitive for large graphs, especially in caching them in memory for real-time online querying. In this paper, we propose a distributed framework that strikes a better balance between offline indexing and online querying. The offline indexing attains a fingerprint of the PPR vector of each vertex by performing billions of "short" random walks in parallel across a cluster of machines. We prove that our indexing method has an exponential convergence, achieving the same precision with previous methods using a much smaller number of random walks. At query time, the new PPR vector is composed by a linear combination of related fingerprints, in a highly efficient vertex-centric decomposition manner. Interestingly, the resulting PPR vector is much more accurate than its offline counterpart because it actually uses more random walks in its estimation. More importantly, we show that such decomposition for a batch of queries can be very efficiently processed using a shared decomposition. Our implementation, PowerWalk, takes advantage of advanced distributed graph engines and it outperforms the state-of-the-art algorithms by orders of magnitude. Particularly, it responses to tens of thousands of queries on graphs with billions of edges in just a few seconds.

show abstract

VENUS: Vertex-centric streamlined graph computation on a single PC

Cited by 82 publications

References 18 publications

GraphMP: I/O-Efficient Big Graph Analytics on a Single Commodity Machine

GraphMP: I/O-Efficient Big Graph Analytics on a Single Commodity Machine

Streaming Temporal Graphs: Subgraph Matching

PowerWalk

Contact Info

Product

Resources

About