Nucleus Decomposition in Probabilistic Graphs: Hardness and Algorithms

Esfahani, Fatemeh; Srinivasan, Venkatesh; Thomo, Alex; Wu, Kui

doi:10.1109/icde53745.2022.00021

Cited by 3 publications

(6 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For pairs of vertices with a number of neighbours in the few hundreds, which is where DP starts being impractical, our CLT approach produces approximations that are indistinguishable from numbers produced by DP. We give theoretical bounds on the quality of the approximation using the Berry-Essen Theorem [12,32].…”

Section: List Of Figuresmentioning

confidence: 99%

“…From the techniques point of view, we utilize the Lyapunov CLT which has not been used before for the problem of clustering probabilistic graphs. Lyapunov CLT has been used in [8,12,11,10] for problems unrelated to clustering, such as core and truss decomposition, non-linear system control, and further applications relevant to social science and economics. The modelling of those problems in terms of Lyapunov CLT does not bear any similarity to our approach.…”

Section: Related Workmentioning

confidence: 99%

“…for some δ > 0, then 1 sn n k=1 (ξ k − µ k ) converges in distribution to a standard normal random variable [12,32].…”

Section: Structural Similarity Using Lyapunov Cltmentioning

confidence: 99%

“…In this section we bound the error of Normal Distribution approximation in equation 4.30 by using the Berry-Essen Theorem [12,32].…”

Section: Approximation Boundmentioning

confidence: 99%

See 3 more Smart Citations

Scaling Up Structural Clustering to Large Probabilistic Graphs Using Lyapunov Central Limit Theorem

Howie,

Srinivasan,

Thomo

2023

Proc. VLDB Endow.

View full text Add to dashboard Cite

Structural clustering is one of the most widely used graph clustering frameworks. In this paper, we focus on structural clustering of probabilistic graphs, which comes with significant computational challenges and has, so far, resisted efficient solutions that are able to scale to large graphs, e.g. the state-of-art can only handle graphs with a few million edges. We address the main bottleneck step of probabilistic structural clustering, computing the structural similarity of vertices based on their Jaccard similarity over the set of possible worlds of a given probabilistic graph. The state-of-art used Dynamic Programming, a quadratic run-time algorithm, that does not scale to pairs of vertices of high degree. In this paper we present a novel approach based on Lyapunov Central Limit Theorem. By using a carefully chosen set of random variables we are able to cast the computation of structural similarity to computing a one-tailed area under the Normal Distribution. Our approach has linear runtime as opposed to quadratic, and as such, it scales to much larger inputs. Extensive experiments show that our approach can handle massive graphs at web-scale which the state-of-art cannot.

show abstract

Section: List Of Figuresmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…for some δ > 0, then 1 sn n k=1 (ξ k − µ k ) converges in distribution to a standard normal random variable [12,32].…”

Section: Structural Similarity Using Lyapunov Cltmentioning

confidence: 99%

“…In this section we bound the error of Normal Distribution approximation in equation 4.30 by using the Berry-Essen Theorem [12,32].…”

Section: Approximation Boundmentioning

confidence: 99%

See 2 more Smart Citations

Scaling Up Structural Clustering to Large Probabilistic Graphs Using Lyapunov Central Limit Theorem

Howie,

Srinivasan,

Thomo

2023

Proc. VLDB Endow.

View full text Add to dashboard Cite

show abstract

“…More recently, Sariyüce generalizes the 𝑟 -cliques and 𝑠-cliques in nucleus decomposition to any pair of subgraphs [47]. There has also been work on nucleus decomposition in probabilistic graphs [18].…”

Section: Related Workmentioning

confidence: 99%

Theoretically and practically efficient parallel nucleus decomposition

Shi¹,

Dhulipala²,

Shun³

2021

Proc. VLDB Endow.

View full text Add to dashboard Cite

This paper studies the nucleus decomposition problem, which has been shown to be useful in finding dense substructures in graphs. We present a novel parallel algorithm that is efficient both in theory and in practice. Our algorithm achieves a work complexity matching the best sequential algorithm while also having low depth (parallel running time), which significantly improves upon the only existing parallel nucleus decomposition algorithm (Sariyüce et al. , PVLDB 2018). The key to the theoretical efficiency of our algorithm is a new lemma that bounds the amount of work done when peeling cliques from the graph, combined with the use of a theoretically-efficient parallel algorithms for clique listing and bucketing. We introduce several new practical optimizations, including a new multi-level hash table structure to store information on cliques space-efficiently and a technique for traversing this structure cache-efficiently. On a 30-core machine with two-way hyper-threading on real-world graphs, we achieve up to a 55x speedup over the state-of-the-art parallel nucleus decomposition algorithm by Sariyüce et al. , and up to a 40x self-relative parallel speedup. We are able to efficiently compute larger nucleus decompositions than prior work on several million-scale graphs for the first time.

show abstract

Theoretically and Practically Efficient Parallel Nucleus Decomposition

Shi

Dhulipala

Shun

2021

Preprint

View full text Add to dashboard Cite

This paper studies the nucleus decomposition problem, which has been shown to be useful in finding dense substructures in graphs. We present a novel parallel algorithm that is efficient both in theory and in practice. Our algorithm achieves a work complexity matching the best sequential algorithm while also having low depth (parallel running time), which significantly improves upon the only existing parallel nucleus decomposition algorithm (Sariyüce et al., PVLDB 2018). The key to the theoretical efficiency of our algorithm is a new lemma that bounds the amount of work done when peeling cliques from the graph, combined with the use of a theoretically-efficient parallel algorithms for clique listing and bucketing. We introduce several new practical optimizations, including a new multi-level hash table structure to store information on cliques space-efficiently and a technique for traversing this structure cache-efficiently. On a 30-core machine with two-way hyper-threading on real-world graphs, we achieve up to a 55x speedup over the state-of-the-art parallel nucleus decomposition algorithm by Sariyüce et al., and up to a 40x self-relative parallel speedup. We are able to efficiently compute larger nucleus decompositions than prior work on several million-scale graphs for the first time.

show abstract

Nucleus Decomposition in Probabilistic Graphs: Hardness and Algorithms

Cited by 3 publications

References 61 publications

Scaling Up Structural Clustering to Large Probabilistic Graphs Using Lyapunov Central Limit Theorem

Scaling Up Structural Clustering to Large Probabilistic Graphs Using Lyapunov Central Limit Theorem

Theoretically and practically efficient parallel nucleus decomposition

Theoretically and Practically Efficient Parallel Nucleus Decomposition

Contact Info

Product

Resources

About