Dynamic Partitioning Supporting Load Balancing for Distributed RDF Graph Stores

Abstract

Various resource description framework (RDF) partitioning methods have been studied for the efficient distributed processing of a large RDF graph. The RDF graph has symmetrical characteristics because subject and object can be used interchangeably if predicate is changed. This paper proposes a dynamic partitioning method of RDF graphs to support load balancing in distributed environments where data insertion and change continue to occur. The proposed method generates clusters and subclusters by considering the usage frequency of the RDF graph that are used by queries as the criteria to perform graph partitioning. It creates a cluster by grouping RDF subgraphs with higher usage frequency while creating a subcluster with lower usage frequency. These clusters and subclusters conduct load balancing by using the mean frequency of queries for the distributed server and conduct graph data partitioning by considering the size of the data stored in each distributed server. It also minimizes the number of edge-cuts connected to clusters and subclusters to minimize communication costs between servers. This solves the problem of data concentration to specific servers due to ongoing data changes and additions and allows efficient load balancing among servers. The performance results show that the proposed method significantly outperforms the existing partitioning methods in terms of query performance time in a distributed server. This paper is organized as follows. Section 2 describes the existing graph partitioning methods. Section 3 describes the RDF dynamic partitioning method proposed in this paper. Section 4 describes the results of the performance evaluation to show the excellence of the proposed method. Finally, Section 5 provides the conclusions of this paper and presents future directions. Related WorkHendrickson and Leland proposed a multilevel method for graph partitioning within acceptable execution time [23]. The multilevel partitioning method generates a sequence of smaller graphs approximating the original graph to find easily a good partition via coarsening procedure. In the coarsening step, it merges two vertices connected to an edge and the new vertex retains edges to the union of the neighbors of the merged vertices. The smallest graph is partitioned via a spectral partitioner. The spectral partitioner needs one, two, or three eigenvectors of the Laplacian matrix to partition a graph into two, four, or eight groups, respectively. The partitioning algorithm is able to handle edge and vertex weights, even if the original graph is unweighted. The multilevel partitioning method makes the sums of the vertex weights in each server as equal as possible and the sum of the weights of edges crossing between servers is minimized.Wang and Chiu proposed a RDF graph partitioning method to minimize the number of edges on different servers [40]. This method first scans the RDF graph and stores subgraph with common vertices into the same components. If the connected components of the graph are found, each connected componen...

Figures of Graph Partitioning by Counting, Sequence and Layer Matrices

“…In the same context the minimum gap graph partitioning problem was formulated, as addressed in [18]. Partition strategies on resource description framework graphs have been studied in [19] and in [20]. Graph contraction (creating of a graph minor [21]) is used in some specific graph-related problems [22].…”

“…The classifiers discussed above seem perfectly fit for this task (like Table 10 contains a summary for Figure 1 as the exemplified case). 4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27, 28} U2, D2: {1, 2, 3,4,6,7,9,10,11,12,13,14,15,16,18,19,20,21,22,24,25,26,27 Regarding the pairing (U2, D2) appearing for C 28 − T d (Figure 27), this pairing does not appear for C 28 − D 2 (Figure 25 vs. Figure 26) suggesting that its ...…”

A Workload-Adaptive Streaming Partitioner for Distributed Graph Stores

“…Although vertex-centric partitioners promote locality, they may severely impact the computational load balance for power-law graphs. 5 In other words, by grouping all edges of high-degree vertices together, a subset of nodes are overloaded. On the other hand, edge-centric methods tend to assign the edges incident to a particular vertex into different partitions.…”