Triangle counting for scale-free graphs at scale in distributed memory

Pearce, Roger

doi:10.1109/hpec.2017.8091051

Cited by 52 publications

(46 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in Fig. 9, we see that MEGA with θ * outperforms the algorithm in [39] on different real-world networks by averagely 2.97 times faster. Therefore, we conclude that if we can find an optimal threshold θ * that leverages the structure of the graph in advance, then MEGA can complete the counting task in pruning with a computational time complexity of O(N ) and beat the algorithm in [39].…”

Section: Evaluation On Real-world Networkmentioning

confidence: 92%

“…We compare MEGA with the algorithm in [39], which is an award-winning work of the MIT/Amazon/IEEE Graph Challenge [14]. The algorithm in [39] assigns direction to each edge based on the degree of each vertex. It implies that, for each vertex v, there are deg + (v) , where |E(G)| is the time complexity of the direction assignment and deg + max is the maximum outdegree.…”

Section: Evaluation On Real-world Networkmentioning

confidence: 99%

See 1 more Smart Citation

MEGA: Machine Learning-Enhanced Graph Analytics for Infodemic Risk Management

Hang

Ling³

et al. 2020

Preprint

View full text Add to dashboard Cite

Statistical network analysis plays a critical role in managing the coronavirus disease (COVID-19) infodemic such as addressing community detection and rumor source detection problems in social networks. As the data underlying infodemiology are fundamentally huge graphs and statistical in nature, there are computational challenges to the design of graph algorithms and algorithmic speedup. A framework that leverages cloud computing is key to designing scalable data analytics for infodemic control. This paper proposes the MEGA framework, which is a novel joint hierarchical clustering and parallel computing technique that can be used to process a variety of computational tasks in large graphs. Its unique feature lies in using statistical machine learning to exploit the inherent statistics of data to accelerate computation. Our MEGA framework consists of first pruning, followed by hierarchical clustering based on geodesic distance and then parallel computing, lending itself readily to parallel computing software, e.g., MapReduce or Hadoop. In particular, we illustrate how our MEGA framework computes two representative graph problems for infodemic control, namely network motif counting for community detection and network centrality computation for rumor source detection. Interesting special cases of optimal tuning in the MEGA framework are identified based on geodesic distance characterization and random graph model analysis. Finally, we evaluate its performance using cloud software implementation and real-world graph datasets to demonstrate its computational efficiency over existing state of the art.

show abstract

Section: Evaluation On Real-world Networkmentioning

confidence: 92%

Section: Evaluation On Real-world Networkmentioning

confidence: 99%

MEGA: Machine Learning-Enhanced Graph Analytics for Infodemic Risk Management

Hang

Ling³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Here, • denotes element-wise multiplication III. COMMUNITY SUBMISSIONS Graph Challenge 2017 received 22 submissions by 111 authors from 36 organizations [24]- [31], [34]- [41], [48]- [53]. The submissions were judged by a panel of experts on their effectiveness at using Graph Challenge to highlight innovations in graph algorithms, hardware, software, and systems.…”

Section: Triangle Countingmentioning

confidence: 99%

GraphChallenge.org: Raising the Bar on Graph Analytic Performance

Samsi¹,

Gadepally²,

Hurley³

et al. 2018

2018 IEEE High Performance Extreme Computing Conference (HPEC)

View full text Add to dashboard Cite

The rise of graph analytic systems has created a need for new ways to measure and compare the capabilities of graph processing systems. The MIT/Amazon/IEEE Graph Challenge has been developed to provide a well-defined community venue for stimulating research and highlighting innovations in graph analysis software, hardware, algorithms, and systems. GraphChallenge.org provides a wide range of preparsed graph data sets, graph generators, mathematically defined graph algorithms, example serial implementations in a variety of languages, and specific metrics for measuring performance. Graph Challenge 2017 received 22 submissions by 111 authors from 36 organizations. The submissions highlighted graph analytic innovations in hardware, software, algorithms, systems, and visualization. These submissions produced many comparable performance measurements that can be used for assessing the current state of the art of the field. There were numerous submissions that implemented the triangle counting challenge and resulted in over 350 distinct measurements. Analysis of these submissions show that their execution time is a strong function of the number of edges in the graph, Ne, and is typically proportional to N 4/3 e for large values of Ne. Combining the model fits of the submissions presents a picture of the current state of the art of graph analysis, which is typically 10 8 edges processed per second for graphs with 10 8 edges. These results are 30 times faster than serial implementations commonly used by many graph analysts and underscore the importance of making these performance benefits available to the broader community. Graph Challenge provides a clear picture of current graph analysis systems and underscores the need for new innovations to achieve high performance on very large graphs.

show abstract

“…The algorithm is based on the asynchronous visitor queue approach presented in the paper and the algorithm is similar to PSP algorithm we discussed above. An extension of this work is presented at the static graph challenge [23].…”

Section: Related Workmentioning

confidence: 99%

Distributed, Shared-Memory Parallel Triangle Counting

Kanewala

Zalewski

Lumsdaine

2018

Proceedings of the Platform for Advanced Scientific Computing Conference

View full text Add to dashboard Cite

Triangles are the most basic non-trivial subgraphs. Triangle counting is used in a number of different applications, including social network mining, cyber security, and spam detection. In general, triangle counting algorithms are readily parallelizable, but when implemented in distributed, shared-memory, their performance is poor due to high communication, imbalance of work, and the difficulty of exploiting locality available in shared memory. In this paper, we discuss four different (but related) triangle counting algorithms and how their performance can be improved in distributed, shared-memory by reducing in-node load imbalance, improving cache utilization, minimizing network overhead, and minimizing algorithmic work. We generalize the four different triangle counting algorithms into a common framework and show that for all four algorithms the in-node load imbalance can be minimized while utilizing caches by partitioning work into blocks of vertices, the network overhead can be minimized by aggregation of blocks of work, and algorithm work can be reduced by partitioning vertex neighbors by degree.We experimentally evaluate the weak and the strong scaling performance of the proposed algorithms with two types of synthetic graph inputs and three real-world graph inputs. We also compare the performance of our implementations with the distributed, shared-memory triangle counting algorithms available in PowerGraph-GraphLab and show that our proposed algorithms outperform those algorithms, both in terms of space and time.

show abstract

Triangle counting for scale-free graphs at scale in distributed memory

Cited by 52 publications

References 9 publications

MEGA: Machine Learning-Enhanced Graph Analytics for Infodemic Risk Management

MEGA: Machine Learning-Enhanced Graph Analytics for Infodemic Risk Management

GraphChallenge.org: Raising the Bar on Graph Analytic Performance

Distributed, Shared-Memory Parallel Triangle Counting

Contact Info

Product

Resources

About