Replication, Load Balancing and Efficient Range Query Processing in DHTs

Pitoura, T.; Ntarmos, Nikos; Triantafillou, Peter

doi:10.1007/11687238_11

Cited by 77 publications

(76 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In the following, we plan to demonstrate the effectiveness of our protocol to minimize overloaded peers and create a load-balanced image of the system. In order to give a metric that summarizes the quality of load balancing achieved, we use the Gini coefficient, as it has been proposed in [9].…”

Section: Resultsmentioning

confidence: 99%

Measuring the cost of online load-balancing in distributed range-queriable systems

Konstantinou

Tsoumakos

Koziris

2009

2009 IEEE Ninth International Conference on Peer-to-Peer Computing

View full text Add to dashboard Cite

Distributed systems such as Peer-to-Peer overlays have been shown to efficiently support the processing of range queries over large numbers of participating hosts. In such systems, uneven load allocation has to be effectively tackled in order to minimize overloaded peers and optimize their performance. In this work, we detect and analyze the two basic methodologies used to achieve load-balancing: Iterative key re-distribution between neighbors and node migration. Based on this analysis, we propose a hybrid method that adaptively utilizes these two extremes to achieve both fast and cost-effective load-balancing in distributed systems that support range queries. As a case study, we offer an implementation on top of a Skip Graph, where we validate our findings in a variety of workloads. Our experimental analysis shows that the hybrid method converges 10% faster than simple neighbor item exchanges and is more than 70% bandwidth efficient compared to simple node migrations.

show abstract

Section: Resultsmentioning

confidence: 99%

Measuring the cost of online load-balancing in distributed range-queriable systems

Konstantinou

Tsoumakos

Koziris

2009

2009 IEEE Ninth International Conference on Peer-to-Peer Computing

View full text Add to dashboard Cite

show abstract

“…However, a data replication system must enforce the consistency among multiple copies of data, and retrieve data efficiently from multiple sites. Previous works mainly focus on how to minimize the response time of retrieving data [16], [17] and balance the load of machines storing the replications [15], [14], [13]. In contrast we consider a more general and complicated problem in which the replicated data have preferences to each other.…”

Section: Related Workmentioning

confidence: 99%

“…Data replication has been extensively studied in distributed system [13], [14], [15], [16], [17], [18]. However, we are not aware of any discussion on data replication of a graph.…”

Section: Introductionmentioning

confidence: 99%

Data Replication for Distributed Graph Processing

Liu

2013

2013 IEEE Sixth International Conference on Cloud Computing

View full text Add to dashboard Cite

Abstract-We present a data replication framework for distributed graph processing. First we partition a graph and store each partition in a machine. Then we replicate all partitions and assign replicas to machines, where each machine can store only a limited number of replicas. The goal is to replicate the partitions so that each partition has at least a certain number of replicated copies, and the cost is minimized. The cost is defined as the data traffic needed to run general graph processing algorithms. The cost metric is the overall transmission cost of all machines, and the maximum transmission cost of a single machine. We propose an optimal algorithm based on linear programming to solve the problem of minimizing the overall transmission cost. We also propose an optimal algorithm to solve a special problem of minimizing the maximum transmission cost of a node.

show abstract

“…Sahin et al [24] improved the previous approach by allowing exact rangequery solutions. Pitoura et al [13] attack simultaneously the problems of providing efficient range query processing and data-access load balance. Their solution uses a hash function which is both locality-preserving and randomised.…”

Section: Related Workmentioning

confidence: 99%

“…the extension of DHT capabilities to support order-based and aggregation queries. DHT-based solutions can be classified into those which modify the DHT [9,6,12,13] and those which leave it unchanged [14,15]. The former group adapts the DHT so that complex queries can be performed.…”

Section: Introductionmentioning

confidence: 99%