Dynamic data replication and placement strategy in geographically distributed data centers

Abstract

With the evolution of geographically distributed data centers in the Cloud Computing landscape along with the amount of data being processed in these data centers, which is growing at an exponential rate, processing massive data applications become an important topic. Since a given task may require many datasets for its execution and the datasets are spread over several different data centers, finding an efficient way to manage the datasets storage across nodes of a Cloud system is a difficult problem.In fact, the execution time of a task might be influenced by the cost of data transfers, which mainly depends on two criterias. The first one is the initial placement of the input datasets during the build-time phase, while the second is the replication of the datasets during the runtime phase. The replication is explicitly consider when datasets are being migrated over the data centers in order to make them locally available wherever needed. Data placement and data replication are important challenges in Cloud Computing. Nevertheless, many studies focus on data placement or data replication exclusively. In this paper, a combination of a data placement strategy followed by a dynamic data replication management strategy is proposed, with the purpose of reducing the associated cost of all data transfers between the (distant) data centers. Our proposed data placement approach considers the main characteristics of a data center such as storage capacity and read/write speeds to efficiently store the datasets, while our dynamic data replication management approach considers three parameters: the number of replicas in the system, the dependency between datasets and tasks and the storage capacity of data centers. The decision of when and whether to keep or to delete replicas is determined by the fulfillment of those three parameters. Our approach estimates the total execution time of the tasks as well as the monetary cost, considering the data transfers activity. Our experiments are conducted using Cloudsim simulator. The obtained results show that our proposed strategies produce an efficient data management by reducing the overheads of the data transfers, compared to both a data placement without replication (by 76%) and the selected data replication approach from Kouidri et al. (by 52%), and by improving the financial cost.

Online Task Scheduling of Big Data Applications in the Cloud Environment

“…It is important to note that data placement and replication techniques were investigated in many papers such as [66][67][68]. These papers can not be compared to the scheduling algorithms reviewed in our related work section, but we mention them because they study the importance of data availability in the cloud computing environment.…”

“…• b ij is an integer value that represents the bandwidth of the connection between machines m i and m j (MB/second). • β ij is the elementary data transfer time [68] between machines m i and m j , it is defined by:…”

“…(2) The remote use is when the required dataset is stored in a different node than the one hosting the task; in that case, data migration is needed from a distant source. We can clearly see that due to the migration process, the execution time of the consumer task is affected by adding a data migration time DMT, where DMT ij is the time to migrate all the datasets required by t i from their locations to m j (m j is also where t i is assigned) [68].…”

“…We would like to mention that in the case where the storage of m j is not enough to store the required datasets of t i , we select a set of datasets to delete from m j . For that, we use our previous work [68], based on data replication, for data selection and deletion processes. The idea is based on two factors:…”

A Novel Data Management Scheme in Cloud for Micromachines

“…The node that is estimated with the smallest workload in the future is chosen to place data blocks. A dynamic data replication strategy is proposed in [20] to minimize network bandwidth consumption and latency. The authors consider a set of characteristics, such as number of replicas, dependency between datasets, and the storage capacity to decide the lifetime of a replica.…”