Scaling graph traversal to 281 trillion edges with 40 million cores

Cao, Huanqi; Wang, Yuanwei; Wang, Haojie; Lin, Hua; Ma, Zixuan; Yin, Wen; Chen, Wenguang

doi:10.1145/3503221.3508403

Cited by 10 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Next, we compare the results from our model to a real example of a highly-scalable graph analytics system. We use our model to estimate the performance of the new Sunway supercomputer running BFS on a synthetic graph with 17.56 trillion vertices and 281 trillion edges and compare our result with the performance reported by Cao et al [6].…”

Section: Use Case 3: System Evaluationmentioning

confidence: 94%

“…As the size of real-world graphs increases, future computing systems must support largescale graph processing in a cost-effective and timely manner. Today's largest benchmark graphs are on the order of 100 billion edges and 3 billion vertices [3], and some systems are processing graphs with over 200 trillion edges [6]. In addition to this enormous scale, graph processing has many challenges, such as frequent random memory accesses and low spatial and temporal locality [8].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Model for Scalable and Balanced Accelerators for Graph Processing

Fariborz

Samani

O'Neill

et al. 2022

IEEE Comput. Arch. Lett.

View full text Add to dashboard Cite

Designing a graph processing system that can scale to graph sizes that are orders of magnitude larger than what is possible on a single accelerator requires a careful codesign of accelerator memory bandwidth and capacity, the interconnect bandwidth between accelerators, and the overall system architecture. We present a high-level bottleneck-analysis model for design and evaluation of scalable and balanced accelerators for graph processing. We show several applications of this model including how to choose the right mix of different memory types, network topology, network bisection bandwidth, and system-level architecture to match the access patterns and capacity requirements of different data structures for a given graph and a performance target.

show abstract

Section: Use Case 3: System Evaluationmentioning

confidence: 94%

Section: Introductionmentioning

confidence: 99%

A Model for Scalable and Balanced Accelerators for Graph Processing

Fariborz

Samani

O'Neill

et al. 2022

IEEE Comput. Arch. Lett.

View full text Add to dashboard Cite

show abstract

“…Therefore, academics and big technology companies like Facebook, Google, and Microsoft have proposed different solutions for organizing and analyzing the rising prevalence of big graphs [12]. Furthermore, the size of these graphs has rapidly increased, with hundreds of billions of nodes and trillions of edges being possible [13], [14]. As the graph size scales up, graph analysis can be performed in a distributed environment.…”

Section: Introductionmentioning

confidence: 99%

“…As the graph size scales up, graph analysis can be performed in a distributed environment. However, graph computing has become a challenging problem due to access irregularity, lack of locality, and intrinsic load imbalance distribution of graphs in different computing clusters [14]. Thus, researchers highlight the critical role of design computing systems in our society today [12].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Graph Computing Systems and Partitioning Techniques: A Survey

et al. 2022

View full text Add to dashboard Cite

Graphs are a tremendously suitable data representation that models the relationships of entities in many application domains, such as recommendation systems, machine learning, computational biology, social network analysis, and other application domains. Graphs with many vertices and edges have become quite prevalent in recent years. Therefore, graph computing systems with integrated various graph partitioning techniques have been envisioned as a promising paradigm to handle large-scale graph analytics in these application domains. However, scalable processing of large-scale graphs is challenging due to their high volume and inherent irregular structure of the real-world graphs. Hence, industry and academia have recently proposed graph partitioning and computing systems to efficiently process and analyze large-scale graphs. The graph partitioning and computing systems have been designed to improve scalability issues and reduce processing time complexity. This paper presents an overview, classification, and investigation of the most popular graph partitioning and computing systems. The various methods and approaches of graph partitioning and diverse categories of graph computing systems are presented. Finally, we discuss future challenges and research directions in graph partitioning and computing systems.

show abstract

Optimising Queries for Pattern Detection Over Large Scale Temporally Evolving Graphs

Chaudhry,

Rossi

2024

IEEE Access

View full text Add to dashboard Cite

Scaling graph traversal to 281 trillion edges with 40 million cores

Cited by 10 publications

References 24 publications

A Model for Scalable and Balanced Accelerators for Graph Processing

A Model for Scalable and Balanced Accelerators for Graph Processing

Graph Computing Systems and Partitioning Techniques: A Survey

Optimising Queries for Pattern Detection Over Large Scale Temporally Evolving Graphs

Contact Info

Product

Resources

About