JA-BE-JA: A Distributed Algorithm for Balanced Graph Partitioning

Abstract

Abstract-Balanced graph partitioning is a well known NPcomplete problem with a wide range of applications. These applications include many large-scale distributed problems including the optimal storage of large sets of graph-structured data over several hosts-a key problem in today's Cloud infrastructure. However, in very large-scale distributed scenarios, state-of-the-art algorithms are not directly applicable, because they typically involve frequent global operations over the entire graph. In this paper, we propose a fully distributed algorithm, called JA-BE-JA, that uses local search and simulated annealing techniques for graph partitioning. The algorithm is massively parallel: there is no central coordination, each node is processed independently, and only the direct neighbors of the node, and a small subset of random nodes in the graph need to be known locally. Strict synchronization is not required. These features allow JA-BE-JA to be easily adapted to any distributed graph-processing system from data centers to fully distributed networks. We perform a thorough experimental analysis, which shows that the minimal edge-cut value achieved by JA-BE-JA is comparable to state-of-the-art centralized algorithms such as METIS. In particular, on large social networks JA-BE-JA outperforms METIS, which makes JA-BE-JA-a bottomup, self-organizing algorithm-a highly competitive practical solution for graph partitioning.

Boosting Vertex-Cut Partitioning for Streaming Graphs

“…Ja-be-Ja [28], [29] is a fully distributed algorithm that uses local search and simulated annealing techniques [30] for graph partitioning. Sheep [31] is a distributed graph partitioner that reduces the graph to an elimination tree, partitions the tree, and then translates the tree partitions into graph partitions.…”

“…Ja-be-Ja-vc [39] is a recent distributed vertex-cut partitioning algorithm, inspired by Ja-be-Ja [28], [29] for edgecut partitioning. Similar to Ja-be-Ja, this algorithm uses local search and simulated annealing to iteratively improve initial random assignment of edges to partitions.…”

A Distributed Algorithm for Large-Scale Graph Partitioning

“…JA-BE-JA [Rahimian et al 2013] is a recent algorithm that is fully distributed and uses local search and SA techniques [Talbi 2009] for graph partitioning. In this algorithm, each vertex is processed independently, and only the direct neighbors of the vertex and a small subset of random vertices in the graph need to be known locally.…”

High quality graph partitioning

“…Hence, in this chapter we try to focus on selected related work and recent coordinate free techniques that have not yet been covered by these papers. There are further methods like space-filling curves [130,131,76,149], methods that handle graphs having geometrical information [22,76,82,119,134], simulated annealing [92] and ant-colony optimization [103,44] or parallel approaches to graph partitioning [94,167,164,41,42,87,136] which are not discussed here. Moreover, at this place we restrict ourselves to the techniques that we did not integrate into our systems or only present an abstract view.…”

“…Basic properties of these graphs can be found in Chapter 2.4. sult of the evolutionary algorithm. Using this methodology KaFFPaStrong, KaFFPaEco, KaFFPaFast, KaFFPaE, Metis and Scotch were able to solve 136,150,170,130,146 and 110 instances respectively. The resulting points achieved in the Pareto challenge can be found in Table 7.7.…”