Hot Random Off-Loading: A Hybrid Storage System with Dynamic Data Migration

Lin, Lin; Zhu, Yan; Yue, Jianhui; Cai, Zhao; Segee, Bruce

doi:10.1109/mascots.2011.41

Cited by 29 publications

(26 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While SLASSD [7] uses an opportunistic goal oriented block I/O scheduling algorithm, Kim et al [2] proposes host level SSD I/O schedulers, which are extensions of state-of-the-art I/O scheduling scheme CFQ. ParDispatcher [49] tries to utilize the parallelism in SSDs, by dividing the entire SSD into sub-regions, each having [6] , SLASSD [7], Axboe [44], Hystor [34], PDC [43], ParDispatcher [49], BFQ [50], FlexDrive [12], AD [28], Borg [8] ADLAM [33], SUORA [4], hatS [21], PDC [43], HRO [45], RPAC [46], RAF [47], PASS [48], Triple-H [11], ExaPlan [20], HybridStore [51], Hystor [34], Scarlett [52], DUX [1] a different queue for dispatching requests. ParDipatcher might be good in applications which have more random I/Os otherwise, leading to increasing wait queues for popular sub-regions and bias in performance.…”

Section: Block Layer Developments Mostly I/o Schedulingmentioning

confidence: 99%

“…ADLAM [33] proposes an adaptable data migration model based on the heat of data to determine the next hot data. HRO [45] migrates or allocates files to SSD based on hotness (access frequency), randomness and profit-value based on read/write-intensiveness and recency of file access. PDC [43] keeps blocks in SSD with highest hit frequency.…”

Section: Multi-tier Storagementioning

confidence: 99%

See 1 more Smart Citation

Host managed contention avoidance storage solutions for Big Data

Mishra

Somani

2017

J Big Data

View full text Add to dashboard Cite

The performance gap between compute and storage is fairly considerable. This results in a mismatch between the application needs from storage and what storage can deliver. The full potential of storage devices cannot be harnessed till all layers of I/O hierarchy function efficiently. Despite advanced optimizations applied across various layers along the odyssey of data access, the I/O stack still remains volatile. The problems associated due to the inefficiencies in data management get amplified in Big Data shared resource environments. The Linux OS (host) block layer is the most critical part of the I/O hierarchy, as it orchestrates the I/O requests from different applications to the underlying storage. Unfortunately, despite it's significance, the block layer, essentially the block I/O scheduler, hasn't evolved to meet the needs of Big Data. We have designed and developed two contention avoidance storage solutions, collectively known as "BID: Bulk I/O Dispatch" in the Linux block layer specifically to suit multitenant, multi-tasking shared Big Data environments. Hard disk drives (HDDs) form the backbone of data center storage. The data access time in HDDs is majorly governed by disk arm movements, which usually occurs when data is not accessed sequentially. Big Data applications exhibit evident sequentiality but due to the contentions amongst other I/O submitting applications, the I/O accesses get multiplexed which leads to higher disk arm movements. BID schemes aim to exploit the inherent I/O sequentiality of Big Data applications to improve the overall I/O completion time by reducing the avoidable disk arm movements. In the first part, we propose a dynamically adaptable block I/O scheduling scheme BID-HDD for disk based storage. BID-HDD tries to recreate the sequentiality in I/O access in order to provide performance isolation to each I/O submitting process. Through trace driven simulation based experiments with cloud emulating MapReduce benchmarks, we show the effectiveness of BID-HDD which results in 28-52% lesser time for all I/O requests than the best performing Linux disk schedulers. In the second part, we propose a hybrid scheme BID-Hybrid to exploit SCM's (SSDs) superior random performance to further avoid contentions at disk based storage. BID-Hybrid is able to efficiently offload non-bulky interruptions from HDD request queue to SSD queue using BID-HDD for disk request processing and multi-q FIFO architecture for SSD. This results in performance gain of 6-23% for MapReduce workloads when compared to BID-HDD and 33-54% over best performing Linux scheduling scheme. BID schemes as a whole is aimed to avoid contentions for disk based storage I/Os following system constraints without compromising SLAs.

show abstract

Section: Block Layer Developments Mostly I/o Schedulingmentioning

confidence: 99%

Section: Multi-tier Storagementioning

confidence: 99%

Host managed contention avoidance storage solutions for Big Data

Mishra

Somani

2017

J Big Data

View full text Add to dashboard Cite

show abstract

“…It monitors the access type and frequency of each file and places frequently read files on SSDs and frequently written files on mechanical disks, so as to separate read and write I/O requests. Hot random off‐loading (HRO) extends this concept by considering not only the access frequency of files but also their random‐access characteristics . Specifically, it monitors the hotness and randomness of each individual file.…”

Section: Backround and Related Workmentioning

confidence: 99%

“…Nevertheless, this problem is similar to the classic 0/1 knapsack problem, which in turn is NP‐complete . Thus, we develop an approximation algorithm called Profit Caching, which heuristically calculates the benefit value of each block and selects only the high‐benefit value blocks to cache in flash memory.…”

Section: Profit Caching: Identifying High‐profit Blocksmentioning

confidence: 99%

Profit data caching and hybrid disk-aware Completely Fair Queuing scheduling algorithms for hybrid disks

Chang

Liao

Chang

et al. 2014

Softw. Pract. Exper.

View full text Add to dashboard Cite

SUMMARYRecently, a hybrid disk drive that integrates a small amount of flash memory within a mechanical drive has received significant attention. The hybrid drive extends the storage hierarchy by using flash memory to cache data from the mechanical disk. Unfortunately, current caching architectures fail to fully exploit the potential of the hybrid drive. Furthermore, current disk input/output (I/O) schedulers are optimized for rotational mechanical disk drives and thus must be re-targeted for the hybrid disk drive. In this paper, we propose a new data caching scheme, called Profit Caching, for hybrid drives. Profit Caching is a self-optimizing caching algorithm. It considers and seamlessly integrates all possible data characteristics that impact the performance of hybrid drives, including read count, write count, sequentiality, randomness, and recency, to determine the caching policy. Moreover, we propose a hybrid disk-aware Completely Fair Queuing (HA-CFQ) scheduler to avoid unnecessary I/O anticipations of the CFQ scheduler. We have implemented Profit Caching and HA-CFQ scheduler in the Linux kernel. Coupled with a trace-driven simulator, we have also conducted detailed experiments under a variety of workloads. Experimental results show that Profit Caching provides significantly improved performance compared with the previous schemes. In particular, the throughput of Profit Caching outperforms previous Random Access First and FlashCache caching schemes by factors of up to 1.8 and 7.6, respectively. In addition, the HA-CFQ scheduler reduces the total execution time of the CFQ scheduler by up to 1.74%. Finally, the experimental results show that the runtime overhead of Profit Caching is extremely insignificant and can be ignored.

show abstract

“…Some wear leveling operations are done to well balance the write loads of different parts of a SSD[15][16][17]. So we just care about the total write load of SSDs without worrying about which part of a SSD a video should be written in.…”

mentioning

confidence: 99%

A feedback-based adaptive data migration method for hybrid storage VOD caching systems

Ling

Yan

et al. 2014

Multimed Tools Appl

View full text Add to dashboard Cite

Nowadays Video-On-Demand (VOD) caching systems are often equipped with hybrid storage devices, which have been designed to combine the high read speed of Solid State Disks (SSDs) and the large capacity of Hard Disk Drives (HDDs). However, the number of erase cycles of SSDs is limited. So it is important to control the write load of SSDs in real applications. This paper proposes a Feedback-based Adaptive Data Migration (FADM) method, which can utilize the real-time feedback of the write load of SSDs to adjust the rule of moving data between HDDs and SSDs. More specifically, a video in HDDs is allowed to be moved into SSDs when its popularity is higher than that of the least popular video in SSDs by a threshold. This threshold is adaptively adjusted according to the feedback of the write load of SSDs. With FADM, the desired lifetime of SSDs can be well guaranteed even under various user behaviors while good read performance can be maintained. Simulations are done to demonstrate the effectiveness of FADM.

show abstract

Hot Random Off-Loading: A Hybrid Storage System with Dynamic Data Migration

Cited by 29 publications

References 10 publications

Host managed contention avoidance storage solutions for Big Data

Host managed contention avoidance storage solutions for Big Data

Profit data caching and hybrid disk-aware Completely Fair Queuing scheduling algorithms for hybrid disks

A feedback-based adaptive data migration method for hybrid storage VOD caching systems

Contact Info

Product

Resources

About