Load Distribution Fairness in P2P Data Management Systems

Abstract

the "poor metrics", the "offline metrics", and the "localoptima" problems. We address the issue of measuring storage, or queryIn the related literature one can find plenty of metrics load distribution fairness in peer-to-peer data for load balancing purposes. However, these metrics are management systems. Existing metrics may look only used to evaluate specific load balancing algorithms, promising from the point of view of specific peers, while and they convey poor information regarding the actual in reality being far from optimal from a global load distribution among nodes. For example, the load perspective. Thus, first we define the requirements and imbalance ratio ([11) measures the load ratio of the study the appropriateness of various statistical metrics heaviest to the lightest node in the network; the 99.9th for measuring load distribution fairness towards these percentile node utilization ([13]) expresses the maximum requirements. The metric proposed as most appropriate is node utilization of 99.9% of its nodes. They may capture the Gini coefficient (G). Second, we develop novel the existence of a small number of overloaded nodes, but distributed sampling algorithms to compute G on-line, none of them manages to capture the fairness of load with high precision, efficiently, and scalably. Third, we distribution from a global perspective which accounts for show how G can readily be utilized on-line by higherthe load in all nodes. Thus, what is very much lacking in level algorithms which can now know when to best the community is the definition of appropriate metrics intervene to correct load imbalances. Our analysis and naturally capturing the notion of fair load distribution and experiments testify for the efficiency and accuracy of providing rich information about this distribution. This is, these algorithms, permitting the online use of a rich and what we call, the "poor metrics" problem. reliable metric, conveying a global perspective of the Furthermore, and perhaps more importantly, such distribution. metrics should not be simply utilized offline, i.e., simply test the efficacy of specific algorithms after the fact (i.e., 1. Introduction after they ran) but, they should also guide the functionality offered by a specific algorithm online, One of the main challenges in peer-to-peer (P2P) helping the algorithm to achieve fair load distribution. networks is how to efficiently manage load distribution This formulates the "offline metrics" problem.(either data access or storage load) around the network to Finally, with respect to the approaches for load avoid overloaded nodes and achieve uniformity. P2P balancing in the literature, there is a lack of a global networks based on distributed hash tables (DHT) attempt perspective when actions are taken to correct load to balance load using cryptographic hashes to randomize imbalances. Corrective actions are always based solely on the mapping between data items and nodes. However local information. Thus, they may look promising from e m d e the point of view of a...

Distributed hash sketches

“…We thus further instrumented FreePastry to report on the aforesaid metrics; namely node insertion hits and node query hits. In order to visualize the impact of the different approaches on load distribution, we employed the Gini Coefficient [Damgaard and Weiner 2000], as advocated in Pitoura and Triantafillou [2007]. In brief, Pitoura and Triantafillou [2007] compare the appropriateness and performance of nine of the most well-known distribution fairness metrics (including the standard deviation, skewness, kurtosis, coefficient of variation, 2:34 • N. Ntarmos et al maximum-to-minimum load ratio, fairness index, Lorenz curves, and the Gini Coefficient) for P2P applications.…”

Pyracanthus: A scalable solution for DHT-independent content-based publish/subscribe data networks

“…Lorenz curves and the Gini coefficient have been recently proposed as an appropriate metric to describe load imbalances in p2p systems [37]. Lorenz curves are functions of the cumulative percentage of ordered items mapped onto the corresponding cumulative percentage of their size.…”

Multi-dimensional data density estimation in P2P networks

“…For example, load balancing [27,36] is an important problem in P2P networks. Dynamical data items and skewed user query patterns in P2P systems may cause some of the peers to become bottlenecks, thereby resulting in severe load imbalance and consequently increasing user response time.…”