Counting triangles in data streams

Abstract

We present two space bounded random sampling algorithms that compute an approximation of the number of triangles in an undirected graph given as a stream of edges. Our first algorithm does not make any assumptions on the order of edges in the stream. It uses space that is inversely related to the ratio between the number of triangles and the number of triples with at least one edge in the induced subgraph, and constant expected update time per edge. Our second algorithm is designed for incidence streams (all edges incident to the same vertex appear consecutively). It uses space that is inversely related to the ratio between the number of triangles and length 2 paths in the graph and expected update time O(log |V | · (1 + s · |V |/|E|)), where s is the space requirement of the algorithm. These results significantly improve over previous work [20,8]. Since the space complexity depends only on the structure of the input graph and not on the number of nodes, our algorithms scale very well with increasing graph size and so they provide a basic tool to analyze the structure of large graphs. They have many applications, for example, in the discovery of Web communities, the computa- * This work was partially supported by the EU within the 6th Framework Programme under contract 001907 "Dynamically Evolving, Large Scale Information Systems" (DELIS) * Part of this work was done while the author was post-doc at Universitá degli Studi di Roma "La Sapienza" † Part of this work was done while the author was visiting the School of Computer Science at Carnegie Mellon UniversityPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. tion of clustering and transitivity coefficient, and discovery of frequent patterns in large graphs.We have implemented both algorithms and evaluated their performance on networks from different application domains. The sizes of the considered graphs varied from about 8, 000 nodes and 40, 000 edges to 135 million nodes and more than 1 billion edges. For both algorithms we run experiments with parameter s = 1, 000, 10, 000, 100, 000, 1, 000, 000 to evaluate running time and approximation guarantee. Both algorithms appear to be time efficient for these sample sizes. The approximation quality of the first algorithm was varying significantly and even for s = 1, 000, 000 we had more than 10% deviation for more than half of the instances. The second algorithm performed much better and even for s = 10, 000 we had an average deviation of less than 6% (taken over all but the largest instance for which we could not compute the number of triangles exactly).

Colorful triangle counting and a MapReduce implementation

“…For many applications, especially in the context of large social networks, an exact count is not crucial but rather a fast, high quality estimate. Most of the work on approximate triangle counting is sampling-based and has considered a (semi-)streaming setting [5,6,7,14,23]. A different line of research is based on a linear algebraic approach [4,21].…”

Spectral Counting of Triangles in Power-Law Networks via Element-Wise Sparsification

“…Z.Bar-Yossef, Kumar and Sivakumar showed in [4] how one can approximate the number of triangles by using the AlonMatias-Szegedy ( [2]) method for approximating frequency moments. New streaming algorithms were introduced in [6].…”

Practical verified computation with streaming interactive proofs

“…Estimating the number of triangles in a graph has received significant attention because of its relevance to database query optimization-knowing the degree of transitivity of a relation is useful when estimating the cost of evaluation plans for certain relational queriesand investigating structural properties of the web-graph and social graphs [11,25,67]. In the absence of annotation, any single-pass algorithm to determine if there is a non-zero number of triangles requires Ω(n 2 ) bits of space, where n is the number of vertices in the graph [11].…”