Snow White Clouds and the Seven Dwarfs

Abstract

Abstract-With increasing availability of Cloud computing services, this paper addresses the challenge consumers of Infrastructure-as-a-Service (IaaS) have in determining which IaaS provider and resources are best suited to run an application that may have specific Quality of Service (QoS) requirements. Utilising application modelling to predict performance is an attractive concept, but is very difficult with the limited information IaaS providers typically provide about the computing resources. This paper reports on an initial investigation into using Dwarf benchmarks to measure the performance of virtualised hardware, conducting experiments on BonFIRE and Amazon EC2. The results we obtain demonstrate that labels such as 'small', 'medium', 'large' or a number of ECUs are not sufficiently informative to predict application performance, as one might expect. Furthermore, knowing the CPU speed, cache size or RAM size is not necessarily sufficient either as other complex factors can lead to significant performance differences. We show that different hardware is better suited for different types of computations and, thus, the relative performance of applications varies across hardware. This is reflected well by Dwarf benchmarks and we show how different applications correlate more strongly with different Dwarfs, leading to the possibility of using Dwarf benchmark scores as parameters in application models.

Predicting Application Performance for Multi-vendor Clouds Using Dwarf Benchmarks

“…One con guration uses a spherical cut-o for the electrostatic calculations and the other one uses the Particle Mesh Ewald (PME) method. We observe in [3] that these algorithms do correlate di erently with the di erent Dwarfs, although computing an approximation of the same physical property.…”

“…This can be a very time consuming and costly exercise, which typically leads to relying on a single IaaS provider. Dwarf benchmarks have been proposed as a way to describe compute resources in a uniform manner across di erent IaaS providers whilst also being intended to be su ciently information-rich to be used directly in application modelling [3]. This paper demonstrates how the Dwarf benchmarks can be used in application modelling to successfully predict the performance of several common multimedia and scienti c applications.…”

“…A generic application model takes as input a description of the expected static application workload, a description of the resources (physical or virtual) used to execute the application (including the resource reliability) and a description of any expected user interactions which contribute to the workload or otherwise a ect the process [3]. Using a mathematical process, the model makes a prediction of the application performance.…”

“…The Dwarf taxonomy was furthered developed at UC Berkeley [7,8] Initial results in [3] indicate that using this taxonomy of Dwarfs is a useful way to describe Cloud compute resources as they expose non-obvious di erences in resources deemed to be the same by the IaaS provider.…”

“…Similarly to Phillips et al [3] we make use of the following three applications For Gromacs, two di erent workloads have been chosen. One con guration uses a spherical cut-o for the electrostatic calculations and the other one uses the Particle Mesh Ewald (PME) method.…”