FatPaths: Routing in Supercomputers and Data Centers when Shortest Paths Fall Short

Besta, Maciej; Schneider, Marcel; Cynk, Karolina; Konieczny, Marek T.; Henriksson, Erik; Girolamo, Salvatore Di; Singla, Ankit; Hoefler, Torsten

doi:10.48550/arxiv.1906.10885

Cited by 3 publications

(3 citation statements)

References 80 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Third, while there exists past research into the impact of the underlying network on the performance of a distributed graph analytics framework [132], little was done into investigating this performance relationship in the context of graph database workloads. To the best of our knowledge, there are no efforts into developing a topology-aware or routing-aware data distribution scheme for graph databases, especially in the context of recently proposed data center and high-performance computing network topologies [24,103] and routing architectures [30,75,111].…”

Section: Challengesmentioning

confidence: 99%

Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph Queries

Besta¹,

Peter²,

Gerstenberger³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Graph processing has become an important part of multiple areas of computer science, such as machine learning, computational sciences, medical applications, social network analysis, and many others. Numerous graphs such as web or social networks may contain up to trillions of edges. Often, these graphs are also dynamic (their structure changes over time) and have domain-specific rich data associated with vertices and edges. Graph database systems such as Neo4j enable storing, processing, and analyzing such large, evolving, and rich datasets. Due to the sheer size of such datasets, combined with the irregular nature of graph processing, these systems face unique design challenges. To facilitate the understanding of this emerging domain, we present the first survey and taxonomy of graph database systems. We focus on identifying and analyzing fundamental categories of these systems (e.g., triple stores, tuple stores, native graph database systems, or object-oriented systems), the associated graph models (e.g., RDF or Labeled Property Graph), data organization techniques (e.g., storing graph data in indexing structures or dividing data into records), and different aspects of data distribution and query execution (e.g., support for sharding and ACID). 45 graph database systems are presented and compared, including Neo4j, OrientDB, or Virtuoso. We outline graph database queries and relationships with associated domains (NoSQL stores, graph streaming, and dynamic graph algorithms). Finally, we describe research and engineering challenges to outline the future of graph databases.

show abstract

Section: Challengesmentioning

confidence: 99%

Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph Queries

Besta¹,

Peter²,

Gerstenberger³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Moreover, contrarily to static graph processing, little research exists into accelerating streaming graph processing using hardware acceleration such as FPGAs [26], [38], [59], high-performance networking hardware and associated abstractions [64], [31], [27], [181], [28], [87], low-cost atomics [165], [182], hardware transactions [30], and others [27], [10]. One could also investigate topology-aware or routing-aware data distribution for graph streaming, especially together with recent high-performance network topologies [29], [130] and routing [37], [144], [88]. Finally, ensuring speedups due to different data modeling abstractions, such as the algebraic abstraction [126], [33], [34], [136], may be a promising direction.…”

Section: Challengesmentioning

confidence: 99%

Practice of Streaming Processing of Dynamic Graphs: Concepts, Models, and Systems

Besta¹,

Fischer²,

Kalavri³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Graph processing has become an important part of various areas of computing, including machine learning, medical applications, social network analysis, computational sciences, and others. A growing amount of the associated graph processing workloads are dynamic, with millions of edges added or removed per second. Graph streaming frameworks are specifically crafted to enable the processing of such highly dynamic workloads. Recent years have seen the development of many such frameworks. However, they differ in their general architectures (with key details such as the support for the parallel execution of graph updates, or the incorporated graph data organization), the types of updates and workloads allowed, and many others. To facilitate the understanding of this growing field, we provide the first analysis and taxonomy of dynamic and streaming graph processing. We focus on identifying the fundamental system designs and on understanding their support for concurrency and parallelism, and for different graph updates as well as analytics workloads. We also crystallize the meaning of different concepts associated with streaming graph processing, such as dynamic, temporal, online, and time-evolving graphs, edge-centric processing, models for the maintenance of updates, and graph databases. Moreover, we provide a bridge with the very rich landscape of graph streaming theory by giving a broad overview of recent theoretical related advances, and by analyzing which graph streaming models and settings could be helpful in developing more powerful streaming frameworks and designs. We also outline graph streaming workloads and research challenges.

show abstract

“…The sizes of such datasets will continue to grow; Sogou Corp. expects a ≈60 trillion edge graph dataset with whole-web crawling. Lowering the size of such graphs is increasingly important for academia and industry: It would offer speedups by reducing the number of expensive I/O operations, the amount of data communicated over the network [19,21,29] and by storing a larger fraction of data in caches.…”

Section: Introductionmentioning

confidence: 99%

Slim Graph: Practical Lossy Graph Compression for Approximate Graph Processing, Storage, and Analytics

Besta,

Weber,

Gianinazzi

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

We propose Slim Graph: the first programming model and framework for practical lossy graph compression that facilitates high-performance approximate graph processing, storage, and analytics. Slim Graph enables the developer to express numerous compression schemes using small and programmable compression kernels that can access and modify local parts of input graphs. Such kernels are executed in parallel by the underlying engine, isolating developers from complexities of parallel programming. Our kernels implement novel graph compression schemes that preserve numerous graph properties, for example connected components, minimum spanning trees, or graph spectra. Finally, Slim Graph uses statistical divergences and other metrics to analyze the accuracy of lossy graph compression. We illustrate both theoretically and empirically that Slim Graph accelerates numerous graph algorithms, reduces storage used by graph datasets, and ensures high accuracy of results. Slim Graph may become the common ground for developing, executing, and analyzing emerging lossy graph compression schemes.

show abstract

FatPaths: Routing in Supercomputers and Data Centers when Shortest Paths Fall Short

Cited by 3 publications

References 80 publications

Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph Queries

Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph Queries

Practice of Streaming Processing of Dynamic Graphs: Concepts, Models, and Systems

Slim Graph: Practical Lossy Graph Compression for Approximate Graph Processing, Storage, and Analytics

Contact Info

Product

Resources

About