Dynamic Load Balancing on Non-homogeneous Clusters

Naiouf, Marcelo; Giusti, Laura Cristina De; Chichizola, Franco; Giusti, Armando Eduardo De

doi:10.1007/11942634_8

Cited by 1 publication

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Naiouf et al discussed dynamic and static balancing on heterogeneous cluster architectures. Three forms of load distribution (Direct Static, Predictive Static, and Dynamic by Demand) were studied and analyzed.…”

Section: Related Workmentioning

confidence: 99%

“…In real‐world applications, workloads changes dynamically making it very difficult to predict which data will become hotspot data and which will not . Thus it is necessary to adjust the load dynamically while the application is being executed . We employ a smart approach to dynamically migrate data replicas that is robust to workload changes which occur due to natural variations in data popularity across different periods of time.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An adaptive replica placement approach for distributed key‐value stores

Filho

Cavalcante

Moreira

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

The use of distributed key-value stores (KVS) has experienced fast adoption by various applications in recent years due to key advantages such as hypertext transfer protocol-based RESTful application programming interface, high availability and elasticity. Due to great scalability characteristics, KVS systems commonly use consistent hashing as data placement mechanism. Although KVS systems offer many advantages, they were not designed to dynamically adapt to changing workloads which often include data access skew. Furthermore, the underlying physical storage nodes may be heterogeneous and do not expose their performance capabilities to higher level data placement layers. In this paper, we address those issues and propose an essential step toward a dynamic autonomous solution by leveraging deep reinforcement learning.We design a self-learning approach that incrementally changes the data placement, improving the load balancing. Our approach is dynamic in the sense that is capable of avoiding hot spots, that is, overloaded storage nodes when facing different workloads.Also, we design our solution to be pluggable. It assumes no previous knowledge of the storage nodes capabilities, thus different KVS deployments may make use of it. Our experiments show that our method performs well on changing workloads including data access skew aspects. We demonstrate the effectiveness of our approach through experiments in a distributed KVS deployment. K E Y W O R D Sconsistent hashing, deep reinforcement learning, key-value stores, load balancing, replica placement INTRODUCTIONDistributed key-value stores (KVS) are a well-established approach for cloud data-intensive applications 1 mainly because they are capable of successfully managing huge data traffic driven by the explosive growth of different applications such as social networks, e-commerce, and enterprise.In this work, the focus is on a particular type of KVS, also known as Object Store, which can store and serve any type of data (eg, photo, image, and video). 2 Object Store such as Dynamo 3 and OpenStack-Swift 4 have become widely accepted due to its scalability, high capacity, cost-effective storage and reliable REST programming interface. These systems take advantage of peer-to-peer architecture 5 and replication techniques in order to guarantee high availability and scalability. The data placement of KVS systems are commonly based on a distributed hash table (DHT) and consistent hashing (CHT) 6 with virtual nodes. Consistent hashing is a type of hashing that minimizes the amount of data that needs to move when adding or removing storage nodes. Using only the hash of the id of the data one can determine exactly where that data should be. This mapping of hashes to locations is usually known as "ring. " While this strategy provides item-balancing guarantees, it may be not very efficient in balancing the actual workload of the system for the following reasons.Concurrency Computat Pract Exper. 2020;32:e5675. wileyonlinelibrary.com/journal/cpe

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%