Standards for graph algorithm primitives

Mattson, Tim; Bader, David A.; Berry, J.; Buluç, Aydın; Dongarra, Jack; Faloutsos, Christos; Feo, John; Gilbert, John R.; Gonzalez, Joseph E.; Hendrickson, Bruce; Kepner, Jeremy; Leiserson, Charles E.; Lumsdaine, Andrew; Padua, David; Poole, Stephen W; Reinhardt, Steve; Stonebraker, Mike; Wallach, Steve; Yoo, Andrew S.

doi:10.1109/hpec.2013.6670338

Cited by 84 publications

(33 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar efforts are ongoing for developing standard query languages and JDBC-like interfaces [51] for property graphs, such as the Gremlin language [79] and the efforts to combine openCypher [72], PGQL [78], and G-CORE [9] to create GSQL [43]. There is also ongoing effort to develop a standard set of linear algebra operations for expressing graph algorithms [64].…”

Section: Discussionmentioning

confidence: 99%

The ubiquity of large graphs and surprising challenges of graph processing: extended survey

et al. 2019

View full text Add to dashboard Cite

Graph processing is becoming increasingly prevalent across many application domains. In spite of this prevalence, there is little research about how graphs are actually used in practice. We performed an extensive study that consisted of an online survey of 89 users, a review of the mailing lists, source repositories, and whitepapers of a large suite of graph software products, and in-person interviews with 6 users and 2 developers of these products. Our online survey aimed at understanding: (i) the types of graphs users have; (ii) the graph computations users run; (iii) the types of graph software users use; and (iv) the major challenges users face when processing their graphs. We describe the participants' responses to our questions highlighting common patterns and challenges. Based on our interviews and survey of the rest of our sources, we were able to answer some new questions that were raised by participants' responses to our online survey and understand the specific applications that use graph data and software. Our study revealed surprising facts about graph processing in practice. In particular, real-world graphs represent a very diverse range of entities and are often very large, scalability and visualization are undeniably the most pressing challenges faced by participants, and data integration, recommendations, and fraud detection are very popular applications supported by existing graph software. We hope these findings can guide future research.

show abstract

Section: Discussionmentioning

confidence: 99%

The ubiquity of large graphs and surprising challenges of graph processing: extended survey

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Currently, there is no consensus on what the "building blocks" of graph processing should be. Different programming frameworks use different bases such as sparse matrix operations [11,22], vertex programs from the point of view of a single vertex [8,21], declarative programming [30] or generic task based parallelization [26]. Further, all of the approaches differ in performance, and different frameworks often perform better on different algorithms.…”

Section: Introductionmentioning

confidence: 99%

Navigating the maze of graph analytics frameworks using massive graph datasets

Satish

Sundaram

Patwary

et al. 2014

Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data

163

View full text Add to dashboard Cite

Graph algorithms are becoming increasingly important for analyzing large datasets in many fields. Real-world graph data follows a pattern of sparsity, that is not uniform but highly skewed towards a few items. Implementing graph traversal, statistics and machine learning algorithms on such data in a scalable manner is quite challenging. As a result, several graph analytics frameworks (GraphLab, CombBLAS, Giraph, SociaLite and Galois among others) have been developed, each offering a solution with different programming models and targeted at different users. Unfortunately, the "Ninja performance gap" between optimized code and most of these frameworks is very large (2-30X for most frameworks and up to 560X for Giraph) for common graph algorithms, and moreover varies widely with algorithms. This makes the end-users' choice of graph framework dependent not only on ease of use but also on performance. In this work, we offer a quantitative roadmap for improving the performance of all these frameworks and bridging the "ninja gap". We first present hand-optimized baselines that get performance close to hardware limits and higher than any published performance figure for these graph algorithms. We characterize the performance of both this native implementation as well as popular graph frameworks on a variety of algorithms. This study helps endusers delineate bottlenecks arising from the algorithms themselves vs. programming model abstractions vs. the framework implementations. Further, by analyzing the system-level behavior of these frameworks, we obtain bottlenecks that are agnostic to specific algorithms. We recommend changes to alleviate these bottlenecks (and implement some of them) and reduce the performance gap with respect to native code. These changes will enable end-users to choose frameworks based mostly on ease of use.

show abstract

“…Many graph processing systems are currently under development. These systems are exploring innovations in algorithms [25]- [35], software architecture [36]- [44], software standards [45]- [49], and parallel computing hardware [50]- [56]. The development of novel algorithms and systems to process these data requires the design, generation, and validation of enormous graphs with known properties.…”

Section: Introductionmentioning

confidence: 99%

Design, Generation, and Validation of Extreme Scale Power-Law Graphs

Kepner

Samsi

Arcand

et al. 2018

2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)

Self Cite

View full text Add to dashboard Cite

Massive power-law graphs drive many fields: metagenomics, brain mapping, Internet-of-things, cybersecurity, and sparse machine learning. The development of novel algorithms and systems to process these data requires the design, generation, and validation of enormous graphs with exactly known properties. Such graphs accelerate the proper testing of new algorithms and systems and are a prerequisite for success on real applications. Many random graph generators currently exist that require realizing a graph in order to know its exact properties: number of vertices, number of edges, degree distribution, and number of triangles. Designing graphs using these random graph generators is a time-consuming trial-anderror process. This paper presents a novel approach that uses Kronecker products to allow the exact computation of graph properties prior to graph generation. In addition, when a real graph is desired, it can be generated quickly in memory on a parallel computer with no-interprocessor communication. To test this approach, graphs with 10 12 edges are generated on a 40,000+ core supercomputer in 1 second and exactly agree with those predicted by the theory. In addition, to demonstrate the extensibility of this approach, decetta-scale graphs with up to 10 30 edges are simulated in a few minutes on a laptop.

show abstract

Standards for graph algorithm primitives

Cited by 84 publications

References 4 publications

The ubiquity of large graphs and surprising challenges of graph processing: extended survey

The ubiquity of large graphs and surprising challenges of graph processing: extended survey

Navigating the maze of graph analytics frameworks using massive graph datasets

Design, Generation, and Validation of Extreme Scale Power-Law Graphs

Contact Info

Product

Resources

About