Exploring application and infrastructure adaptation on hybrid grid-cloud infrastructure

Abstract

Clouds are emerging as an important class of distributed computational resources and are quickly becoming an integral part of production computational infrastructures. An important but oft-neglected question is, what new applications and application capabilities can be supported by clouds as part of a hybrid computational platform? In this paper we use the ensemble Kalman-filter based dynamic application workflow and investigate how clouds can be effectively used as an accelerator to address changing computational requirements as well as changing Quality of Service constraints (e.g., deadlines). Furthermore, we explore how application and system-level adaptivity can be used to improve application performance and achieve a more effective utilization of the hybrid platform. Specifically, we adapt the ensemble Kalman-filter based application formulation (serial versus parallel, different solvers etc.) so as to execute efficiently on a range of different infrastructure (from High Performance Computing grids to clouds that support single core and many-core virtual machines). Our results show that there are performance advantages to be had by supporting application and infrastructure level adaptivity. In general, we find that grid-cloud infrastructure can support novel usage modes, such as deadline-driven scheduling, for applications with tunable characteristics that can adapt to varying resource types.

A coordinator for scaling elastic applications across multiple clouds

“…This concept was further extended so that brokers do not represent single users; instead, brokers represent a group of users of some facility that needs to expand resource capacity by buying resources from a Grid or Cloud. This is the case of works by Kim et al [24], Ostermann et al [25], Assunção et al [26], Tordsson et al [27], and Bossche et al [28]. These works focus on policies for the resource consumer, while this work focuses in the resource providers.…”

A Provisioning Approach of Cloud Resources for Dynamic Workflows

“…The elastic nature of cloud environment enables such dynamic workflow to be enacted more efficiently since it facilitates the changing of resource quantities at runtime. According to [16], cloud may be used as an effective accelerator to treat changing computational requirements as well as that of QoS constraints (e.g., deadlines) for a dynamic workflow. Depending on the requirements of the workflow, resources can be added or released at runtime on demand.…”

Towards a Provisioning Algorithm for Dynamic Workflows in the Cloud

“…The elastic nature of cloud environment enables such dynamic workflow to be enacted more efficiently since it facilitates the changing of resource quantities at runtime. According to [12], cloud may be used as an effective accelerator to treat changing computational requirements as well as that of QoS constraints (e.g., deadlines) for a dynamic workflow. Depending on the requirements of the workflow, resources can be added or released at runtime on demand.…”