HDStore: An SSD/HDD Hybrid Distributed Storage Scheme for Large-Scale Data

Feng, Zhijie; Feng, Zhiyong; Wang, Xin; Rao, Guozheng; Wei, Yazhou; Li, Zhiyuan

doi:10.1007/978-3-319-11538-2_20

Cited by 5 publications

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 1 depicts a typical HSS consisting of a fast-yet-small storage device (e.g., [94,95]) and a large-yet-slow storage device (e.g., [96][97][98][99]). Traditional hybrid storage systems [51,100,101] were designed with a smaller NAND flash-based SSD and a larger HDD. Nowadays, hybrid storage systems come with emerging NVM devices (e.g., [102][103][104][105]) coupled with slower high-density NAND flash devices [49,[106][107][108].…”

Section: Background 21 Hybrid Storage Systems (Hsss)mentioning

confidence: 99%

Sibyl: Adaptive and Extensible Data Placement in Hybrid Storage Systems Using Online Reinforcement Learning

Singh¹,

Rakesh²,

Park³

et al. 2022

Preprint

View full text Add to dashboard Cite

Hybrid storage systems (HSS) use multiple different storage devices to provide high and scalable storage capacity at high performance. Data placement across different devices is critical to maximize the benefits of such a hybrid system. Recent research proposes various techniques that aim to accurately identify performance-critical data to place it in a "best-fit" storage device. Unfortunately, most of these techniques are rigid, which (1) limits their adaptivity to perform well for a wide range of workloads and storage device configurations, and (2) makes it difficult for designers to extend these techniques to different storage system configurations (e.g., with a different number or different types of storage devices) than the configuration they are designed for. Our goal is to design a new data placement technique for hybrid storage systems that overcomes these issues and provides: (1) adaptivity, by continuously learning from and adapting to the workload and the storage device characteristics, and (2) easy extensibility to a wide range of workloads and HSS configurations.We introduce Sibyl, the first technique that uses reinforcement learning for data placement in hybrid storage systems. Sibyl observes different features of the running workload as well as the storage devices to make system-aware data placement decisions. For every decision it makes, Sibyl receives a reward from the system that it uses to evaluate the long-term performance impact of its decision and continuously optimizes its data placement policy online.We implement Sibyl on real systems with various HSS configurations, including dual-and tri-hybrid storage systems, and extensively compare it against four previously proposed data placement techniques (both heuristic-and machine learning-based) over a wide range of workloads. Our results show that Sibyl provides 21.6%/19.9% performance improvement in a performanceoriented/cost-oriented HSS configuration compared to the best previous data placement technique. Our evaluation using an HSS configuration with three different storage devices shows that Sibyl

show abstract