Cura: A Cost-Optimized Model for MapReduce in a Cloud

Abstract

Abstract-We propose a new MapReduce cloud service model, Cura, for data analytics in the cloud. We argue that performing MapReduce analytics in existing cloud service models -either using a generic compute cloud or a dedicated MapReduce cloud -is inadequate and inefficient for production workloads. Existing services require users to select a number of complex cluster and job parameters while simultaneously forcing the cloud provider to use those potentially sub-optimal configurations resulting in poor resource utilization and higher cost. In contrast Cura leverages MapReduce profiling to automatically create the best cluster configuration for the jobs so as to obtain a global resource optimization from the provider perspective. Secondly, to better serve modern MapReduce workloads which constitute a large proportion of interactive real-time jobs, Cura uses a unique instant VM allocation technique that reduces response times by up to 65%. Thirdly, our system introduces deadline-awareness which, by delaying execution of certain jobs, allows the cloud provider to optimize its global resource allocation and reduce costs further. Cura also benefits from a number of additional performance enhancements including cost-aware resource provisioning, VMaware scheduling and online virtual machine reconfiguration. Our experimental results using Facebook-like workload traces show that along with response time improvements, our techniques lead to more than 80% reduction in the compute infrastructure cost of the cloud data center.

Fault-Tolerant Resource Provisioning with Deadline-Driven Optimization in Hybrid Clouds

“…To address the problem of operational cost minimization in Cloud resource provisioning context, authors in [9], proposed a new MapReduce Cloud service model called Cura, which automatically creates the best configuration of clusters for tasks so as to approach a global resource optimization. Specifically, the system employs a deadline-awareness method, which defers the execution of certain tasks, and necessitates global optimization with reduced cost.…”

“…Their approach seeks to maximize the profit of the SaaS provider by minimizing the number of SLA violations and the cost by reusing VMs. Other related works [9], [11], [12], [13] and [14] also focused on cost-optimization strategies for resource provisioning. This paper is also focused on cost optimization and also consider workloads that are dynamic in nature.…”

A self-adaptive scheduling algorithm for reduce start time

“…In the early algorithm design, reduce tasks can be scheduled once any map tasks are finished [31]. One of the benefits is that the reduce tasks can copy the output of the map tasks as soon as possible.…”

Towards Operational Cost Minimization in Hybrid Clouds for Dynamic Resource Provisioning with Delay-Aware Optimization

“…With the rapid growth of cloud computing industry, the cloud resource provisioning problem has also attracted many research efforts in cost-optimizing perspective [8], [9], [11], [12], [13], [14], [15], [16].…”