Design, Generation, and Validation of Extreme Scale Power-Law Graphs

Kepner, Jeremy; Samsi, Siddharth; Arcand, William; Bestor, David; Bergeron, Bill; Davis, Tim D.; Gadepally, Vijay; Houle, Michael E.; Hubbell, Matthew; Jananthan, Hayden; Jones, Michael; Klein, Anna; Michaleas, Peter; Pearce, Roger; Milechin, Lauren; Mullen, Julie; Prout, Andrew; Rosa, Antonio; Sanders, Geoffrey; Yee, Charles; Reuther, Albert

doi:10.1109/ipdpsw.2018.00055

Cited by 14 publications

(12 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3. (Self-Loops) As observed in [7], [3], putting self loops into the factors of A and B boosts the number of triangles in C significantly. Therefore we will analyze the case when factors have no self-loops (for simplicity) and cases when one or more of the factors have self loops.…”

Section: Undirected Graphsmentioning

confidence: 80%

“…As shown in [7], [3], it is simple to see the degree distribution vector of C, d C := (C − I C • C)1 C , in terms of the degree distribution vectors of A and B. Without self loops in A and B, (I C • C) = O C , and…”

Section: A Degree-distributionmentioning

confidence: 99%

“…1(b). Note that in the corollary above diag(B 3 ) contains in its k-th entry double counts of triangles diag((B − B • I) 3 ) k and the four other three-step sequences from vertex k to itself that involve non-trivial edges and self loops. For example, if l is connected to k, diag(…”

Section: Proof In Appendixmentioning

confidence: 99%

“…In recent work [3], extremely large synthetic powerlaw Kronecker graphs [2] are generated in an essentially communication-free implementation for the primary purpose of validating graph calculation implementations on benchmarks where the answer is known exactly. The (nonstochastic) Kronecker approach leverages ideas an observations from previous work on stochastic Kronecker graph generators [4]- [6], with the added benefit that the ground truth of many local and global graph statistics are efficiently calculated during the generation process.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

On Large-Scale Graph Generation with Validation of Diverse Triangle Statistics at Edges and Vertices

Sanders¹,

Pearce²,

Fond³

et al. 2018

2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)

Self Cite

View full text Add to dashboard Cite

Researchers developing implementations of distributed graph analytic algorithms require graph generators that yield graphs sharing the challenging characteristics of real-world graphs (small-world, scale-free, heavy-tailed degree distribution) with efficiently calculable ground-truth solutions to the desired output. Reproducibility for current generators [1] used in benchmarking are somewhat lacking in this respect due to their randomness: the output of a desired graph analytic can only be compared to expected values and not exact ground truth. Nonstochastic Kronecker product graphs [2] meet these design criteria for several graph analytics. Here we show that many flavors of triangle participation can be cheaply calculated while generating a Kronecker product graph.Given two medium-sized scale-free graphs with adjacency matrices A and B, their Kronecker product graph has adjacency matrix C = A ⊗ B. Such graphs are highly compressible: |E| edges are represented in O(|E| 1/2 ) memory and can be built in a distributed setting from small data structures, making them easy to share in compressed form. Many interesting graph calculations have worst-case complexity bounds O(|E| p ) and often these are reduced to O(|E| p/2 ) for Kronecker product graphs, when a Kronecker formula can be derived yielding the sought calculation on C in terms of related calculations on A and B.We focus on deriving formulas for triangle participation at vertices, tC , a vector storing the number of triangles that every vertex is involved in, and triangle participation at edges, ∆C , a sparse matrix storing the number of triangles at every edge. When factors A and B are undirected, C is also undirected. In the case when both factors have no self loops we show tC = 2tA ⊗ tB, ∆C = ∆A ⊗ ∆B. Moreover, we derive the respective formulas when A and B have self loops, which boosts the triangle counts for the associated vertices/edges in C. We additionally demonstrate strong assumptions on B that allow the truss decomposition of C to be derived cheaply from the truss decomposition of A.We extend these results and show Kronecker formulas for triangle participation in both directed graphs and undirected, vertex-labeled graphs. In these classes of graphs each vertex / edge can participate in many different types of triangles.

show abstract

Section: Undirected Graphsmentioning

confidence: 80%

Section: A Degree-distributionmentioning

confidence: 99%

Section: Proof In Appendixmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On Large-Scale Graph Generation with Validation of Diverse Triangle Statistics at Edges and Vertices

Sanders¹,

Pearce²,

Fond³

et al. 2018

2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Realistic synthetic models have been used in testing softwareintensive cyber-physical systems in [1,3,18,30,75,76] as well as testing software tools (e.g., design or analysis tools) used for engineering safety-critical systems [14,22,33,37,56,74,90]. For example, realistic test models used for autonomous cars represent test environments [1,30] where unrealistic test cases (e.g., obscure traffic situations) are considered false positives.…”

Section: Problem Statementmentioning

confidence: 99%

Automated generation of consistent, diverse and structurally realistic graph models

et al. 2021

View full text Add to dashboard Cite

In this paper, we present a novel technique to automatically synthesize consistent, diverse and structurally realistic domain-specific graph models. A graph model is (1) consistent if it is metamodel-compliant and it satisfies the well-formedness constraints of the domain; (2) it is diverse if local neighborhoods of nodes are highly different; and (1) it is structurally realistic if a synthetic graph is at a close distance to a representative real model according to various graph metrics used in network science, databases or software engineering. Our approach grows models by model extension operators using a hill-climbing strategy in a way that (A) ensures that there are no constraint violation in the models (for consistency reasons), while (B) more realistic candidates are selected to minimize a target metric value (wrt. the representative real model). We evaluate the effectiveness of the approach for generating realistic models using multiple metrics for guidance heuristics and compared to other model generators in the context of three case studies with a large set of real human models. We also highlight that our technique is able to generate a diverse set of models, which is a requirement in many testing scenarios.

show abstract

A Distributed Block Chebyshev-Davidson Algorithm for Parallel Spectral Clustering

Pang,

Yang

2024

J Sci Comput

View full text Add to dashboard Cite

Design, Generation, and Validation of Extreme Scale Power-Law Graphs

Cited by 14 publications

References 66 publications

On Large-Scale Graph Generation with Validation of Diverse Triangle Statistics at Edges and Vertices

On Large-Scale Graph Generation with Validation of Diverse Triangle Statistics at Edges and Vertices

Automated generation of consistent, diverse and structurally realistic graph models

A Distributed Block Chebyshev-Davidson Algorithm for Parallel Spectral Clustering

Contact Info

Product

Resources

About