Counting triangles in real-world graph streams: Dealing with repeated edges and time windows

Abstract

Real-world graphs often manifest as a massive temporal "stream" of edges. The need for real-time analysis of such large graph streams has led to progress on low memory, one-pass streaming graph algorithms. These algorithms were designed for simple graphs, assuming an edge is not repeated in the stream. Real graph streams however, are almost always multigraphs i.e., they contain many duplicate edges. The assumption of no repeated edges requires an extra pass storing all the edges just for deduplication, which defeats the purpose of small memory algorithms.We describe an algorithm, MG-Triangle, for estimating the triangle count of a multigraph stream of edges. We show that all previous streaming algorithms for triangle counting fail for multigraph streams, despite their impressive accuracies for simple graphs. The bias created by duplicate edges is a major problem, and leads these algorithms astray. MG-Triangle avoids these biases through careful debiasing strategies and has provable theoretical guarantees and excellent empirical performance. MG-Triangle builds on the previously introduced wedge sampling methodology. Another challenge in analyzing temporal graphs is finding the right temporal window size. MG-Triangle seamlessly handles multiple time windows, and does not require committing to any window size(s) a priori. We apply MGTriangle to discover fascinating transitivity and triangle trends in real-world graph streams.

Approximately counting triangles in large graph streams including edge duplicates with a fixed memory usage

“…1 http://nskeylab.xjtu.edu.cn/dataset/phwang/code/PartitionCT.zip In summary, edges in the graph streaming model studied in this paper have: 1) duplicates, i.e., an edge in Π may appear more than once, which is similar to the data streaming model in [19,22]; 2) time-variant edge direction labels, i.e., each edge label l (t) (t) may change with t, which results in existing streaming algorithms failing to approximately count directed triangle patterns over time.…”

“…The algorithms are implemented in C++, and run on a computer with a Quad-Core Intel(R) Xeon(R) CPU E3-1226 v3 CPU 3.30GHz processor. We compare our method PartitionCT with four state-of-the-art methods: TRIÉST [44], MASCOT [30], FURL [22], and MG-Triangle [19]. For TRIÉST, both of its basic TRIÉST-BASE and improved TRIÉST-IMPR variants are considered.…”

“…To handle edge duplicates, one can use a uniform random hash function r to hash each edge e into the range (0, 1), and e is sampled when r(e) ≤ p. This hash-based sampling is essentially equivalent to the method of sampling edges with a fixed probability p, so it has serious issues (e.g., the algorithm may run out of memory) for analyzing large graph streams discussed previously. Moreover, to avoid storing all duplicates of a sampled edge e with r(e) ≤ p in memory, hash-based sampling requires an extra computational cost to check whether a coming edge e has been already sampled when r(e) ≤ p. Using hashbased sampling, Jha et al [19] recently present the first triangle count approximation algorithm for large graph streams including edge duplicates. Besides no guarantee on the amount of memory usage, it also has a high computational cost because it enumerates each sampled wedge to check whether a coming edge is the closing edge or one edge of the wedges.…”

REPT: A Streaming Algorithm of Approximating Global and Local Triangle Counts in Parallel

“…However, GPS requires more memory usage to store the sampling weights of sampled edges, and more runtime for sampling weights calculation and update. [43], [47], [48] develop one-pass streaming algorithms to deal with large graph streams including edge duplications. In detail, Wang et al [43] develop PartitionCT for triangle count approximation with a fixed memory usage, which uses a family of hash functions to uniformly sample distinct edges at a high speed, and this can reduce the sampling cost per edge to O(1) without additional memory usage.…”

“…In detail, Wang et al [43] develop PartitionCT for triangle count approximation with a fixed memory usage, which uses a family of hash functions to uniformly sample distinct edges at a high speed, and this can reduce the sampling cost per edge to O(1) without additional memory usage. Jha et al [47] present MG-TRIANGLE algorithm to estimate the triangle counts in multigraph streams. Jung et al [48] develop FURL to approximate local triangles for all nodes in multigraph streams.…”