Burstiness-aware I/O scheduler for MapReduce framework on virtualized environments

Abstract

Recently, virtualized environments such as cloud computing and a virtual cluster are used popularly by lots of MapReduce applications to reap the benefits of low cost and flexibility. However, the I/O bottleneck of the virtualization software gives a burden especially for processing big data. To relieve the burden, we propose a novel burstiness-aware I/O scheduler. Our analysis has revealed that the I/O bottleneck is caused by I/O interferences among the bursty I/Os triggered by different virtual machines, especially when they execute the map and/or reduce tasks. The I/O interferences result in frequent context switches in the virtualization software and long seek distances in a disk. Our proposed I/O scheduler first detects I/O burstiness of a virtual machine on-line. Then, it schedules bursty virtual machines in a round-robin fashion so that a scheduled virtual machine utilizes most of I/O bandwidth without interferences. Real implementation based experiments haveshown that our scheduler can enhance the I/O performance up to 23% with an average of 20%.