Towards Lightweight and Swift Storage Resource Management in Big Data Cloud Era

Abstract

Workload IO behavior in modern data centers is fluctuating and unpredictable due to the rapidly adopted, public cloud environment. Nevertheless, existing storage resource management systems, such as VMware SDRS, are incapable of performing real time policybased storage management due to the high cost of migrating large size virtual disks. Hence, the traditional storage management schemes become ineffective due to the lack of quick response to the frequent IO bursts and the inaccurate storage latency prediction in the light of a highly fluctuating environment.To address the aforementioned issues, we propose LightSRM, which can work properly in a time-variant cloud environment. To mitigate the storage migration cost, we leverage copy-on-write/read snapshots to redirect the IO requests without moving the virtual disk. To support snapshots in storage management, we also build a performance model specifically for snapshots. We employ exponentially weighted moving average with adjustable sliding window to provide quick and accurate performance prediction. Furthermore, we propose a hybrid management scheme, which can dynamically choose either snapshot or migration for fastest performance tuning.We build our prototype in a QEMU/KVM based virtualized environment. Our empirical evaluation results show that snapshot can redirect IO requests in a faster manner than migration can do when the virtual disk size is large. Besides, snapshot method has less disk performance impact on the applications. By employing hybrid snapshot/migration method, LightSRM yields less overall latency, better load balance, and less IO traffic overhead.

Bridging the Semantic Gaps of GPU Acceleration for Scale-out CNN-based Big Data Processing

“…DjiNN [15] makes the first attempt to explore commodity GPU-based CNN accelerator server platform and provides beneficial implication to future warehouse-scale computer design. However, the enormous amount of data that generated in current IT bignames' warehouse-scale computers present significant challenges for scale-out CNN-based big data processing [18,26,47]. For example, more than 350 million photos are being posted to Facebook per day [13,25] and 100 hours of video are being uploaded to YouTube per minute [42], such daunting amount of data arrival remarkably embarrasses the throughput of traditional standalone CNN accelerators.…”