A Space-Efficient Streaming Algorithm for Estimating Transitivity and Triangle Counts Using the Birthday Paradox

Abstract

We design a space-efficient algorithm that approximates the transitivity (global clustering coefficient) and total triangle count with only a single pass through a graph given as a stream of edges. Our procedure is based on the classic probabilistic result, the birthday paradox. When the transitivity is constant and there are more edges than wedges (common properties for social networks), we can prove that our algorithm requires O( √ n) space (n is the number of vertices) to provide accurate estimates. We run a detailed set of experiments on a variety of real graphs and demonstrate that the memory requirement of the algorithm is a tiny fraction of the graph. For example, even for a graph with 200 million edges, our algorithm stores just 40,000 edges to give accurate results. Being a single pass streaming algorithm, our procedure also maintains a real-time estimate of the transitivity/number of triangles of a graph by storing a minuscule fraction of edges. ACM Reference Format:Madhav Jha, C. Seshadhri, and Ali Pinar. 2015. A space-efficient streaming algorithm for estimating transitivity and triangle counts using the birthday paradox. ACM Trans. Knowl. Discov. Data 9, 3, Article 15 (February 2015), 21 pages.

Tiered sampling: An efficient method for approximate counting sparse motifs in massive graph streams

“…Counting subgraphs in large networks is a well studied problem in data mining which was originally brought to attention in the seminal work of Milo et al [5]. In particular, many contributions in the literature have focused on the triangle counting problem, including exact algorithms, MapReduce algorithms [11,12] and streaming algorithms [6,7,13,14].…”

“…Using a strategy similar to our TieredSampling approach, in [14] Jha and Seshadri propose a one pass streaming algorithm for triangle counting which using a first reservoir for edges which are then used to generate a stream of wedges (i.e., paths of length two) stored in a second reservoir. This approach appear to be not worthwhile for triangle counting as it is consistently outperformed by a simpler strategy based on a single reservoir presented [6].…”

“…If the sub-structure selected as a tool for counting the motif of interest is not "rare enough" then there is no benefit in devoting a certain amount of the memory budget to maintaining a sample of occurrences of the sub-structure. Such problem would for instance arise when using the TieredSampling approach for counting triangles using wedges (i.e., two-hop paths) as a sub-structure, as attempted in [14], as in most real-world graph the number of wedges is much greater of the number of edges themselves making them not suited to be used as a sub-structure.…”

Mining Frequent Patterns in Evolving Graphs

“…Tsourakakis et al [33] proposed triangle sparsifiers to approximate the triangle counts with a single pass of the graph, hence, the technique can also be applied to incremental streams. Pavan et al [28] and Jha et al [20] proposed sampling a set of connected paths of length for approximately counting the triangles in incremental streams. Lim and Kang [27] proposed an algorithm based on Bernoulli sampling of edges for incremental streams, in which the edges are kept in the sample with a fixed user-defined probability.…”

On estimating the Spectral Radius of large graphs through subgraph sampling

“…Biased sampling to make an unbiased estimate of a property of the full graph has recently gained increased attention in the analysis of large graphs. However, its use so far has largely been limited to ascertaining only localized properties of graphs (such as the number of triangles in the graph, the clustering co-efficient or its motif statistics) and not a complex property like the spectral radius [16]- [19].…”