A Declarative Environment for Automatic Performance Evaluation in IaaS Clouds

Abstract

One of the main challenges faced by users of infrastructure-as-a-service (IaaS) clouds is the difficulty to adequately estimate the virtual resources necessary for their applications. Although many cloud providers offer programatic ways to rapidly acquire and release resources, it is important that users have a prior understanding of the impact that each virtual resource type offered by the provider may impose on application performance. This paper presents Cloud Crawler, a new declarative environment aimed at supporting users in describing and automatically executing application performance tests in IaaS clouds. To this end, the environment provides a novel declarative domain-specific language, called Crawl, which supports the description of a variety of performance evaluation scenarios in multiple IaaS clouds; and an extensible Java-based cloud execution engine, called Crawler, which automatically configures, executes and collects the results of each performance evaluation scenario described in Crawl. To illustrate Cloud Crawler's potential benefits, the paper reports on an experimental evaluation of a social network application in two public IaaS cloud providers, in which the proposed environment has successfully been used to investigate the application performance for different virtual machine configurations and under different demand levels.

Bottleneck Detection and Solution Recommendation for Cloud-Based Multi-Tier Application

“…Cloud has gathered pace, as most enterprises are moving toward the agile hosting of multi-tier applications in public clouds, many researchers have focused on three different research directions: 1) updating application architecture to move from legacy systems to clouds [3], 2) evaluating different clouds' functional and non-functional attributes for allowing cloud users to correctly make a decision on which cloud to host applications [2,4,7,8], and 3) efficiently orchestrating virtual appliances in a cloud, which may also include negotiations with cloud users. While some highly related previous work has principally focused on estimating rudimentary cloud capabilities using benchmarks [8] and automated performance testing [9],our approach focuses on the precise characterization of application capabilities in a cloud infrastructure.…”

Recommending Optimal Cloud Configuration based on Benchmarking in Black‐Box Clouds

“…The importance of estimating cloud configuration has been mentioned in (Cunha, 2013) as well, and they have proposed an approach to collect the estimates from public clouds. However, in their approach, users must need a help from application expert to deploy the application and collect estimates.…”

“…In this regard, many researchers have focused on three (Chauhan, 2011), (ii) evaluating different clouds' functional and non-functional attributes for informed decision on which cloud to host applications (Jayasinghe, 2011;Huang, 2010;Calheiros, 2011;Jayasinghe, 2011;Cunha, 2013), and (iii) efficient orchestration of virtual appliances in a cloud (Keahey, 2008), which may also include negotiations with users (Venugopal, 2009). This paper complements these different directions by enabling recommendation of cloud configuration to suit application requirements.…”

PEESOS-Cloud: A Workload-Aware Architecture for Performance Evaluation in Service-Oriented Systems

“…The results showed that the prototype is functional and able to measure the metrics of the target environment. Cunha et al (2013) [13] create a programmable environment to evaluate the performance of IaaS.The environment allows several performance test parameters to be set up, such as the number of providers and their features. A performance evaluation for two levels of workload was conducted using an environment consisting of IaaS EC2 and Rackspace machines.…”