Superset: A Non-uniform Replica Placement Strategy towards High-Performance and Cost-Effective Distributed Storage Service

Abstract

Load balance and power proportionality are both important aspects in constructing high-performance and costeffective distributed storage systems. However, traditional replica placement strategies towards load balance usually produce scattered replica layouts which disable power proportionality, while recent strategies towards power proportionality are typically based on uniform replication which compromises the ability of load balance. In this article, we introduce Superset (an organized non-uniform replica placement strategy) which takes both load balance and power proportionality into consideration. The main idea is to partition the whole system into multiple uniform replication based subsystems with the accommodated file subsets satisfying the 'superset' condition. We have conducted a series of simulations with real-world distributions of data popularity. Our results show that, compared to state of the art solutions, Superset consumes less energy to fulfill the same performance requirement while offers better performance subject to the same energy consumption constraint.

Energy wasting at internet data centers due to fear

“…In such systems, such as Green-HDFS [22], the ultimate goal is to efficiently distribute the machines between these different areas, maximizing the overall performance thanks to improvements in the hot zone, minimizing the overall energy consumption thanks to improvements in the cold zone, increasing the time response as little as possible when reading files from machines switched off (in GreenHDFS, only 2.1% of the readings were affected by this temporary penalty due to switching on the machine at the time of the reading), thereby significantly reducing the energy consumption of servers: 24% in the case of GreenHDFS [23]. • Dynamic replication: Other solutions, such as Superset [27], take the above strategies as a starting point, but also take into account the "temperature" of the data above a threshold, not only to power on/off machines, but also to increase or decrease the number of copies of stored data, thereby preserving the availability of data and reducing overall energy consumption thanks to the switching on/off policies and improved performance. This is achieved by transferring storage space and computing power from the cold files that are not frequently used, to those files that need these resources, i.e, the hottest files.…”

Limiting Global Warming by Improving Data-Centre Software

“…The authors propose an energy-aware scheduling policy based on Dynamic Voltage and Frequency Scaling (DVFS) in [32]. Moreover, various approaches look for the reduction of the energy consumption by applying energyproportionality models based on power-proportional distributed file systems in [33]- [35] which generally aim to switch storage-servers off when the replicated data is not needed.…”

A survey on cost-effective context-aware distribution of social data streams over energy-efficient data centres

“…For Data Replication and Placement to improve energy proportionality, many solutions [3,91,106,145] have been proposed. In [3], authors describe a power-proportional distributed file system which stores copies of data on non-overlapping subsets of resources.…”