Smart CloudBench -- Automated Performance Benchmarking of the Cloud

Chhetri, Mohan Baruwal; Chichin, Sergei; Vo, Quoc Bao; Kowalczyk, Ryszard

doi:10.1109/cloud.2013.7

Cited by 35 publications

(24 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Here, constraint (1) ensures that the demand must not exceed the supply for type k. The constraint (2) confirms that the winners pay at most of their valuation, and at least the price that the provider wants for the requested bundle r j . We define the bundle-specific group price as g.r j / D P k2K r k j p k .t k /, which is the weighted sum of group price for all the requested instances in the bundle, and t k denotes the traded amount of type k, that is, t k D P j 2W r k j .…”

Section: Introductionmentioning

confidence: 91%

See 1 more Smart Citation

SGAM: strategy‐proof group buying‐based auction mechanism for virtual machine allocation in clouds

Zhang

Huang

et al. 2015

Concurrency and Computation

View full text Add to dashboard Cite

We study the cloud resource auction problem where users can bid for resource bundle containing heterogeneous types of virtual machines, and providers allocate virtual machines to their users through group price model. Compared with fixed price model, which is not always the best approach for trading resources as its economically inefficient and inflexible nature, the group price model possess the better flexibility and monetary benefits for auction participants (e.g., cloud providers and users). The proposed auction mechanism strategy-proof group buying-based auction mechanism, which formulates the problem of virtual machine allocation in clouds as a combinatorial auction problem, and holds some important property such as individual rationality, ex-post budget balance, and truthfulness, meanwhile guaranteeing efficiency in both the provider's revenue and system efficiency. Extensive simulation results show that the proposed mechanism yields the allocation efficiency and computational tractability compared with the mechanism with fixed price model. This enables bidders to express their valuations (i.e., private type) for requested bundle in a more meaningful way.There exist many challenges in designing a practical auction-based mechanism for the VM allocation in clouds. We list three major challenges as follows:Strategy-proofness: Because the participants are normally rational and selfish in the auction, they always tend to strategically manipulate the auction, if doing so can increase their utilities. Such strategic behaviors may hurt the other participant's utility. Therefore, designing an auction mechanism, which can prevent the participants from having an incentive to lie about their valuation of the requested bundle, is a critical issue in auction market. Such a mechanism is called strategy-proof mechanism (refer to Section 2 for the definition), simply submitting truthful valuation for the requested bundle maximizes each participant's utility. Allocative efficiency: The objective of an allocative efficient mechanism is to maximize social welfare, which is the sum of the winners' valuations of the allocated instances. Low execution time: Cloud computing is a large-scale and dynamic environment. The auction mechanism for cloud resources has to have a low computation complexity for deciding the winners and the payment in order to be feasible in real cloud market. System modelThe cloud provider (i.e., seller) sets different configurations of physical resources that she wishes to sell to the cloud users (i.e., buyers) in the form of VM instances.We assume that the cloud provider offers K different instance types, K D ¹1; : : : ; Kº, and specifies the amounts q k of supplied VMs of type k.2 K/. Besides, the provider declares the discrete price curve p k .m/ W Z C ! R C for the instance of type k, which indicates the minimum unit price of the supplied instance that the provider wishes to sell, and is a decreasing step function of m, the number of units sold together, that is, f or m 1 < m 2 and m 1 ; m 2 > 0; p k ....

show abstract

Section: Introductionmentioning

confidence: 91%

“…(2) We propose a strategy-proof group buying-based auction mechanism (SGAM), which formulates the problem of VMs allocation and solve the above address challenges. (2) We propose a strategy-proof group buying-based auction mechanism (SGAM), which formulates the problem of VMs allocation and solve the above address challenges.…”

Section: Introductionmentioning

confidence: 99%

SGAM: strategy‐proof group buying‐based auction mechanism for virtual machine allocation in clouds

Zhang

Huang

et al. 2015

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…Smart CloudBench [16] is another advocator to incorporate price as a metric. It also collected a number of performance metrics for TPC-W workload, including average response time, maximum response time, total number of successful interactions and total number of timeouts.…”

Section: Related Workmentioning

confidence: 99%

Benchmarking Private Cloud Performance with User-Centric Metrics

Sun

Hall

et al. 2014

2014 IEEE International Conference on Cloud Engineering

View full text Add to dashboard Cite

Cloud computing is a new paradigm for the delivery of IT services. It has enabled many promising opportunities for features that cannot be easily implemented in traditional IT environments, such as elastic scalability, self-service deployment, resiliency and recovery, and so forth. Benchmarking the cloud requires a welldefined set of cloud performance metrics that should be able to sensitively distinguish the capabilities of cloud systems that enable those features. One way of defining benchmark metrics is based on observations of the internal mechanisms in a cloud. For example, an elasticity evaluation may be based on measuring a resource provisioning interval in the cloud. However, a more meaningful evaluation should be based on user-centric metrics. In this article, we will introduce a set of performance metrics that can be directly measured, calculated and compared by the cloud users, including workload consumers and the users who deploy and manage the workload life cycles. We will also discuss ways to organize the usercentric metrics, with different emphasis, into a benchmark that represents different use cases.

show abstract

“…When the workload is executed centrally, the mischaracterisation caused by the presence of noise occurs as a result of problems such as the overloading of the components responsible for generating the workload, especially when a single machine has to represent many clients [9], [14]. [21] shows how the excessive number of threads created in the instances that carry out load generation have an adverse effect on the generation of the workload.…”

Section: The Problem Statementmentioning

confidence: 99%

PEESOS-Cloud: A Workload-Aware Architecture for Performance Evaluation in Service-Oriented Systems

Ferreira

Nunes

Pereira

et al. 2016

2016 IEEE World Congress on Services (SERVICES)

View full text Add to dashboard Cite

Abstract-It is a challenging task to ensure quality in serviceoriented systems deployed in cloud computing owing to the dynamicity of its environment. Many approaches have been adopted to identify and evaluate bottlenecks and problems in performance. The most common scenario consists of distributed systems that use a workload capable of enabling clients to exploit the target system in different operational conditions. However, one requirement that tends to be overlooked is to determine how the workload is executed, as software and hardware faults can lead to its mischaracterization. In this paper, a number of problems in the workload generation have been identified and summarized. A new architecture, called PEESOS-Cloud, is proposed which allows these services to be evaluated as well as to improve the ability of the workload so that it conforms with its described characteristics. Experiments in a cloud environment were conducted to show how PEESOS-Cloud works and validate its capabilities. Our experiment also showed that the mischaracterization of the workload leads to poor results, whereas an workload-aware implementation leads to a better performance evaluation.

show abstract

Smart CloudBench -- Automated Performance Benchmarking of the Cloud

Cited by 35 publications

References 12 publications

SGAM: strategy‐proof group buying‐based auction mechanism for virtual machine allocation in clouds

SGAM: strategy‐proof group buying‐based auction mechanism for virtual machine allocation in clouds

Benchmarking Private Cloud Performance with User-Centric Metrics

PEESOS-Cloud: A Workload-Aware Architecture for Performance Evaluation in Service-Oriented Systems

Contact Info

Product

Resources

About