Adaptive Data Migration in Multi-tiered Storage Based Cloud Environment

Zhang, Gong; Chiu, Lawrence; Liu, Ling

doi:10.1109/cloud.2010.60

Cited by 43 publications

(26 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In future, it would be interesting to design a system with BID schemes for block level contention management coupled with self-optimizing block re-organization of BORG [8], adaptive data migration policies of ADLAM [33], and replication-management of such as Triple-H [11]. This could solve the issue of workload and cost-aware tiering for large scale data-centers experiencing Big Data workloads.…”

Section: Discussionmentioning

confidence: 99%

“…Based on profiling of workloads, balanced utility value of data usage, the data is managed between the tiers of storage for improved performance. Workload-aware Storage Tiering, or simply Tiering [3,4] is the automatic classification of how data is managed between heterogeneous tiers of storage in enterprise data-center environment [33]. It is vital to develop automated and dynamic tiering solutions to utilize all the tiers of storage.…”

Section: Bid-hybrid: Contention Avoidance Utilizing Multi-tier Architmentioning

confidence: 99%

“…The effects of caching in enterprise platforms in negligible due to the data set size and skewed workload characteristics [23,29], therefore faster SCMs (SSDs) are used as cache. ADLAM [33] proposes an adaptable data migration model based on the heat of data to determine the next hot data. HRO [45] migrates or allocates files to SSD based on hotness (access frequency), randomness and profit-value based on read/write-intensiveness and recency of file access.…”

Section: Multi-tier Storagementioning

confidence: 99%

“…While SLASSD [7] uses an opportunistic goal oriented block I/O scheduling algorithm, Kim et al [2] proposes host level SSD I/O schedulers, which are extensions of state-of-the-art I/O scheduling scheme CFQ. ParDispatcher [49] tries to utilize the parallelism in SSDs, by dividing the entire SSD into sub-regions, each having [6] , SLASSD [7], Axboe [44], Hystor [34], PDC [43], ParDispatcher [49], BFQ [50], FlexDrive [12], AD [28], Borg [8] ADLAM [33], SUORA [4], hatS [21], PDC [43], HRO [45], RPAC [46], RAF [47], PASS [48], Triple-H [11], ExaPlan [20], HybridStore [51], Hystor [34], Scarlett [52], DUX [1] a different queue for dispatching requests. ParDipatcher might be good in applications which have more random I/Os otherwise, leading to increasing wait queues for popular sub-regions and bias in performance.…”

Section: Block Layer Developments Mostly I/o Schedulingmentioning

confidence: 99%

See 3 more Smart Citations

Host managed contention avoidance storage solutions for Big Data

Mishra

Somani

2017

J Big Data

View full text Add to dashboard Cite

The performance gap between compute and storage is fairly considerable. This results in a mismatch between the application needs from storage and what storage can deliver. The full potential of storage devices cannot be harnessed till all layers of I/O hierarchy function efficiently. Despite advanced optimizations applied across various layers along the odyssey of data access, the I/O stack still remains volatile. The problems associated due to the inefficiencies in data management get amplified in Big Data shared resource environments. The Linux OS (host) block layer is the most critical part of the I/O hierarchy, as it orchestrates the I/O requests from different applications to the underlying storage. Unfortunately, despite it's significance, the block layer, essentially the block I/O scheduler, hasn't evolved to meet the needs of Big Data. We have designed and developed two contention avoidance storage solutions, collectively known as "BID: Bulk I/O Dispatch" in the Linux block layer specifically to suit multitenant, multi-tasking shared Big Data environments. Hard disk drives (HDDs) form the backbone of data center storage. The data access time in HDDs is majorly governed by disk arm movements, which usually occurs when data is not accessed sequentially. Big Data applications exhibit evident sequentiality but due to the contentions amongst other I/O submitting applications, the I/O accesses get multiplexed which leads to higher disk arm movements. BID schemes aim to exploit the inherent I/O sequentiality of Big Data applications to improve the overall I/O completion time by reducing the avoidable disk arm movements. In the first part, we propose a dynamically adaptable block I/O scheduling scheme BID-HDD for disk based storage. BID-HDD tries to recreate the sequentiality in I/O access in order to provide performance isolation to each I/O submitting process. Through trace driven simulation based experiments with cloud emulating MapReduce benchmarks, we show the effectiveness of BID-HDD which results in 28-52% lesser time for all I/O requests than the best performing Linux disk schedulers. In the second part, we propose a hybrid scheme BID-Hybrid to exploit SCM's (SSDs) superior random performance to further avoid contentions at disk based storage. BID-Hybrid is able to efficiently offload non-bulky interruptions from HDD request queue to SSD queue using BID-HDD for disk request processing and multi-q FIFO architecture for SSD. This results in performance gain of 6-23% for MapReduce workloads when compared to BID-HDD and 33-54% over best performing Linux scheduling scheme. BID schemes as a whole is aimed to avoid contentions for disk based storage I/Os following system constraints without compromising SLAs.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Bid-hybrid: Contention Avoidance Utilizing Multi-tier Architmentioning

confidence: 99%

Section: Multi-tier Storagementioning

confidence: 99%

Section: Block Layer Developments Mostly I/o Schedulingmentioning

confidence: 99%

See 2 more Smart Citations

Host managed contention avoidance storage solutions for Big Data

Mishra

Somani

2017

J Big Data

View full text Add to dashboard Cite

show abstract

“…Recently, [8], [9], [10], [11], [12], [13], [14], [15] proposed several new techniques (algorithmic or theoretical) to explore the effective data migration in storage systems. For example, [8], [9] investigated the idea of using edge-coloring theory for data migration and achieved a nearoptimal migration plan by using polynomial-time approximation algorithms.…”

Section: Related Workmentioning

confidence: 99%

Live Data Migration for Reducing SLA Violations in Multi-tiered Storage Systems

Tai

Sheng

Yao

et al. 2014

2014 IEEE International Conference on Cloud Engineering

View full text Add to dashboard Cite

Today, the volume of data in the world has been tremendously increased. Large-scaled and diverse data sets are raising new big challenges of storage, process, and query. Tiered storage architectures combining solid-state drives (SSDs) with hard disk drives (HDDs), become attractive in enterprise data centers for achieving high performance and large capacity simultaneously. However, how to best use these storage resources and efficiently manage massive data for providing high quality of service (QoS) is still a core and difficult problem. In this paper, we present a new approach for automated data movement in multi-tiered storage systems, which lively migrates the data across different tiers, aiming to support multiple service level agreements (SLAs) for applications with dynamic workloads at the minimal cost. Trace-driven simulations show that compared to the no migration policy, LMST significantly improves average I/O response times, I/O violation ratios and I/O violation times, with only slight degradation (e.g., up to 6% increase in SLA violation ratio) on the performance of high priority applications.

show abstract

Data Location Optimization for Improvement of Tiered Storage Performance

Hayashi

Komoda

2015

Elect Comm in Japan

View full text Add to dashboard Cite

SUMMARY We propose a new method for tiered storage and evaluate its characteristics. In this method, the fast tier is divided into two areas and the data in each area are managed on the basis of two I/O (input/output) measurement periods. When I/Os to these areas increase, the proposed method allocates the frequently accessed areas to a fast tier in advance, thereby improving storage I/O performance. We evaluate the proposed method by simulating storage I/Os and data migration between tiers. When I/O locality is high or frequently accessed areas rarely move, the proposed method is more effective and improves storage I/O performance by up to 44%.

show abstract

Adaptive Data Migration in Multi-tiered Storage Based Cloud Environment

Cited by 43 publications

References 4 publications

Host managed contention avoidance storage solutions for Big Data

Host managed contention avoidance storage solutions for Big Data

Live Data Migration for Reducing SLA Violations in Multi-tiered Storage Systems

Data Location Optimization for Improvement of Tiered Storage Performance

Contact Info

Product

Resources

About