How a consumer can measure elasticity for cloud platforms

Islam, Sadeka; Lee, Kevin; Fekete, Alan; Liu, Anna

doi:10.1145/2188286.2188301

Cited by 124 publications

(111 citation statements)

References 13 publications

Supporting

Mentioning

105

Contrasting

Unclassified

Order By: Relevance

“…In that case it makes perfect sense to report how far the load can be scaled up. [17,33], or counted the number of SLA breaches [17] during periods of elastic system activity. However, measuring and characterizing elasticity remain activities without theoretical support.…”

Section: Meaningful Metricmentioning

confidence: 99%

“…Even understanding which periods are elastic, that is, distinguishing between normal fluctuations and significant elastic changes in the system behavior, requires advances in the current body of knowledge. Moreover, the proposal in [17,33] relies on the benchmark being able to collect CPU consumption information of the provider system. In some cases these numbers might not be freely available to the consumer.…”

Section: Meaningful Metricmentioning

confidence: 99%

See 1 more Smart Citation

Benchmarking in the Cloud: What It Should, Can, and Cannot Be

Folkerts

Alexandrov

Sachs

et al. 2013

Selected Topics in Performance Evaluation and Benchmarking

View full text Add to dashboard Cite

With the increasing adoption of Cloud Computing, we observe an increasing need for Cloud Benchmarks, in order to assess the performance of Cloud infrastructures and software stacks, to assist with provisioning decisions for Cloud users, and to compare Cloud offerings. We understand our paper as one of the first systematic approaches to the topic of Cloud Benchmarks. Our driving principle is that Cloud Benchmarks must consider end-to-end performance and pricing, taking into account that services are delivered over the Internet. This requirement yields new challenges for benchmarking and requires us to revisit existing benchmarking practices in order to adopt them to the Cloud.

show abstract

Section: Meaningful Metricmentioning

confidence: 99%

Section: Meaningful Metricmentioning

confidence: 99%

Benchmarking in the Cloud: What It Should, Can, and Cannot Be

Folkerts

Alexandrov

Sachs

et al. 2013

Selected Topics in Performance Evaluation and Benchmarking

View full text Add to dashboard Cite

show abstract

“…There seems a lack of approaches to summary measurement of multiple and even overall Cloud service features. For example, Cloud elasticity is related not only to the resource scaling time but also to the resource charging basis (Islam et al, 2012); consequently, it has become a challenge to explicitly quantify the amount of elasticity of a Cloud service (Li et al, 2012b(Li et al, , 2013c. Therefore, to cover and test various service aspects from a holistic view, the current practitioners normally suggest employing benchmark suites for Cloud services evaluation (Iosup et al, 2011;Rabl et al, 2010).…”

Section: Related Workmentioning

confidence: 99%

“…In fact, although it is often useful and convenient to compare alternatives by using a single index (Islam et al, 2012), the overall measurement of Cloud services would be significantly challenging because of the diverse and even conflicting criteria .…”

Section: Related Workmentioning

confidence: 99%

“…In fact, according to the rich research in the evaluation of traditional computer systems, the selection of metrics plays an essential role in evaluation implementations (Li et al, 2012b). Particularly, it is often useful and significant to evaluate Cloud services from a holistic view (Iosup et al, 2011;Rabl et al, 2010), and using single measurement indexwould be helpful and convenient for comparing alternatives and drawing conclusions (Islam et al, 2012). More importantly, a single index of an overall measurement can play a summary Response role in experimental design and analysis (Montgomery, 2009) for evaluating Cloud services.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Boosting Metrics: Measuring Cloud Services From the Holistic Perspective

Li¹,

O’Brien²,

Zhang³

2014

STCC

View full text Add to dashboard Cite

Studies have shown that Cloud services evaluation would be crucial and beneficial for both service customers and providers, and metrics would play a vital role in any evaluation implementation. Considering the numerous and various aspects of Cloud services, a frequent suggestion is to perform evaluation from a holistic view. The currently normal strategy of holistic evaluation is to use a set of metrics along with a suite of benchmarks to conduct separated experiments. Given the separated, diverse, and even possibly conflicting measurement criteria, it could be still hard for customers with such evaluation reports to understand an evaluated Cloud service from a global perspective. Inspired by the boosting approaches to machine learning, we proposed the concept Boosting Metrics to represent all the potential approaches that are able to deliver summary measurement of Cloud services. Essentially, the idea of boosting metrics is to holistically measure Cloud services with concern of service properties, which supplements the strategy of employing benchmark suites that is to holistically evaluate Cloud services with concern of different workloads. This paper introduces two types of preliminary approaches, and unifies a set of sophisticated measurements into the notion of boosting metrics. In particular, we show that boosting metrics can be used as a summary Response for applying experimental design to Cloud services evaluation. Although the concept Boosting Metrics was refined based on our work in the Cloud Computing domain, we believe it can be easily adapted to the evaluation work of other computing paradigms.

show abstract

Joint‐analysis of performance and energy consumption when enabling cloud elasticity for synchronous HPC applications

Righi

Costa

Rodrigues

et al. 2015

Concurrency and Computation

View full text Add to dashboard Cite

Summary A key characteristic of cloud computing is elasticity, automatically adjusting system resources to an application's workload. Both reactive and horizontal approaches represent traditional means to offer this capability, in which rule‐condition‐action statements and upper and lower thresholds occur to instantiate or consolidate compute nodes and virtual machines. Although elasticity can be beneficial for many HPC (high‐performance computing) scenarios, it also imposes significant challenges in the development of applications. In addition to issues related to how we can incorporate this new feature in such applications, there is a problem associated with the performance and resource pair and, consequently, with energy consumption. Further exploring this last difficulty, we must be capable of analyzing elasticity effectiveness as a function of employed thresholds with clear metrics to compare elastic and non‐elastic executions properly. In this context, this article explores elasticity metrics in two ways: (i) the use of a cost function that combines application time with different energy models; (ii) the extension of speedup and efficiency metrics, commonly used to evaluate parallel systems, to cover cloud elasticity. To accomplish (i) and (ii), we developed an elasticity model known as AutoElastic, which reorganizes resources automatically across synchronous parallel applications. The results, obtained with the AutoElastic prototype using the OpenNebula middleware, are encouraging. Considering a CPU‐bound application, an upper threshold close to 70% was the best option for obtaining good performance with a non‐prohibitive elasticity cost. In addition, the value of 90% for this threshold was the best option when we plan an efficiency‐driven execution. Copyright © 2015 John Wiley & Sons, Ltd.

show abstract

How a consumer can measure elasticity for cloud platforms

Cited by 124 publications

References 13 publications

Benchmarking in the Cloud: What It Should, Can, and Cannot Be

Benchmarking in the Cloud: What It Should, Can, and Cannot Be

Boosting Metrics: Measuring Cloud Services From the Holistic Perspective

Joint‐analysis of performance and energy consumption when enabling cloud elasticity for synchronous HPC applications

Contact Info

Product

Resources

About