Fast and Low-cost Search for Efficient Cloud Configurations for HPC Workloads

Abstract

The wide variety of virtual machine types, network configurations, number of instances, among others configuration tweaks, in cloud computing, makes the finding of the best configuration a hard problem. Trying to reduce costs and resource underutilization while achieving acceptable performance can be a hard task even for specialists. Thus, many approaches to find these optimal or almost optimal configurations for a given program were proposed in the literature. Observing the performance of an application in the cloud takes time and money. Therefore, most of the approaches aim not only to find good solutions but also to reduce the search cost. One of those approaches relies on Bayesian Optimization, which analyzes fewer configurations, reducing the search cost while still finding good solutions. Another approach found in the literature is the use of a technique named Paramount Iteration, which enables users to reason about cloud configurations' cost and performance without executing the application to its completion (early-stopping) this approach reduces the cost of each observation. In this work, we show that both techniques can be used together to do fewer and lower-cost observations. We demonstrate that such an approach can recommend solutions that are 1.68x better on average than Random Searching and with a 6x cheaper search.

Design Space Exploration of Heterogeneous Systems Applied to the Cloud Resource Allocation Problem

“…We have performed validation experiments on the MultiExplorer-VM and pairwise comparisons to the PI technique [Rosario et al 2020] and a brute force algorithm that finds optimal VM configurations but takes a long runtime. The validation experiments were based on statistical tests: estimating the power of the pairwise test, testing for normality of samples (Shapiro-Wilk test 9 ), testing for homogeneity of variances (Bartlett 10 and Fligner-Killeen 11 tests ).…”

“…Some authors focused on exploring the search space of instance configurations. Techniques such as random search and network search achieved lower performance when compared to statistical methods such as Bayesian Optimization with Random Forest [Hsu et al 2018][Rosario et al 2020 or Gausian Process [Alipourfard et al 2017]. Those approaches have the benefit of low-cost search by looking at a few configurations.…”

“…Those approaches have the benefit of low-cost search by looking at a few configurations. In [Rosario et al 2020] the search cost is even lower with the adoption of Paramount Interactions.…”

“…• A comprehensive validation and evaluation on a set of applications from the NPB benchmark and on AWS virtual machine configurations. Additionally, we compare the results of the MultiExplorer-VM to the Paramount Interaction Technique [Rosario et al 2020].…”

Multi-Phase Task-Based HPC Applications: Quickly Learning how to Run Fast

“…They use Newton's Method to find the best number of processes to use. In another work [28], the authors used the GP to search for the best homogeneous cloud instance for a given application considering performance and cost. They restrict to very specific (powers of 2 from 1 to 32) subsets of homogeneous nodes.…”