CLOUDLIGHTNING: A Framework for a Self-organising and Self-managing Heterogeneous Cloud

Lynn, Theo; Xiong, Huanhuan; Dong, Dapeng; Momani, Bilal; Gravvanis, George A.; Filelis‐Papadopoulos, Christos K.; Elster, Anne C.; Khan, Malik M.; Tzovaras, Dimitrios; Giannoutakis, Konstantinos M.; Petcu, Dana; Neagul, Marian; Dragon, Ioan; Kuppudayar, Perumal; Natarajan, S.; McGrath, Michael J.; Gaydadjiev, Georgi; Becker, Tobias; Gourinovitch, Anna; Kenny, David J.; Morrison, John P.

doi:10.5220/0005921503330338

Cited by 32 publications

(19 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have also developed, and may continue to improve a high level language from which one can automatically generate parameterized OpenCL code . How to use some of these techniques for enhancing one or more of the use cases in the EU Horizon 2020 CloudLightning project we are a part of will also be investigated . Furthermore, adding more parameters, including advanced new features specific to a given architecture, may be an interesting possibility.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Machine learning‐based auto‐tuning for enhanced performance portability of OpenCL applications

Falch

Elster

2016

Concurrency and Computation

View full text Add to dashboard Cite

Summary Heterogeneous computing, combining devices with different architectures such as CPUs and GPUs, is rising in popularity and promises increased performance combined with reduced energy consumption. OpenCL has been proposed as a standard for programming such systems and offers functional portability. However, it suffers from poor performance portability, because applications must be retuned for every new device. In this paper, we use machine learning‐based auto‐tuning to address this problem. Benchmarks are run on a random subset of the tuning parameter spaces, and the results are used to build a machine learning‐based performance model. The model can then be used to find interesting subspaces for further search. We evaluate our method using five image processing benchmarks, with tuning parameter space sizes up to 2.3 M, using different input sizes, on several devices, including an Intel i7 4771 (Haswell) CPU, an Nvidia Tesla K40 GPU, and an AMD Radeon HD 7970 GPU. We compare different machine learning algorithms for the performance model. Our model achieves a mean relative error as low as 3.8% and is able to find solutions on average only 0.29% slower than the best configuration in some cases, evaluating less than 1.1% of the search space. The source code of our framework is available at https://github.com/acelster/ML‐autotuning.

show abstract

Section: Discussionmentioning

confidence: 99%

“…61 How to use some of these techniques for enhancing one or more of the use cases in the EU Horizon 2020 CloudLightning project we are a part of will also be investigated. 62 Furthermore, adding more parameters, including advanced new features specific to a given architecture, 63…”

Section: Discussionmentioning

confidence: 99%

Machine learning‐based auto‐tuning for enhanced performance portability of OpenCL applications

Falch

Elster

2016

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…The CloudLightning approach is based on three major components, which are further used to build the supporting use-cases [9]. Although most of these components have been extensively researched in previous projects, it is still an outstanding challenge to implement all of them in close relation to each other.…”

Section: The Main Components Of the Solutionmentioning

confidence: 99%

Exposing HPC services in the Cloud: the CloudLightning Approach

Dragan¹,

Fortiş²,

Neagul³

2016

SCPE

Self Cite

View full text Add to dashboard Cite

Nowadays we are noticing important changes in the way High Performance Computing (HPC) providers are dealing with the demand. The growing requirements of modern data-and compute-intensive applications ask for new models for their development, deployment and execution. New approaches related with Big Data, peta-and exa-scale computing are going to dramatically change the design, development and exploitation of highly demanding applications, such as the HPC ones. Due to the increased complexity of these applications and their outstanding requirements which cannot be supported by the classical centralized cloud models, novel approaches, inspired by autonomic computing, are investigated as an alternative. In this paper, we offer an overview of such an approach, undertaken by the CloudLightning initiative [9]. In this context, a novel cloud delivery model that offers the capabilities to describe and deliver dynamic and tailored services is being considered. This new delivery model, based on a self-organizing and self-managing approach, will allow provisioning and delivery of coalitions of heterogeneous cloud resources, built on top of the resources hosted by a cloud service provider.

show abstract

“…The initial implementation and the deployment of the proposed schemes have been realized in the context of CloudLightning project [14]. In this paper, a use case based on the Intel's Ray-Tracing application [9] is used to demonstrate the need for a unified platform to manage a cloud environment composed of heterogeneous resources.…”

Section: Methodsmentioning

confidence: 99%

Heterogeneous Resource Management and Orchestration in Cloud Environments

Dong

Xiong

Castañé

et al. 2018

Cloud Computing and Service Science

Self Cite

View full text Add to dashboard Cite

Abstract. The addition of heterogeneous resources to conventional homogeneous cloud environments has enabled clouds to embrace a wide variety of new applications that heretofore were traditionally confined to specialized computing environments. The enhanced and extended features offered by heterogeneous resources enable service offerings that pose challenges to traditional cloud management throughout the entire service delivery stack. The accelerated uptake of heterogeneous resources is exacerbating these challenges, which no longer can be efficiently addressed in an ad-hoc manner. Therefore, an integrated approach to heterogeneous resource management that is cognizant of the unique advantages of different hardware types is needed. In this paper, two candidate approaches, a platform-integration scheme and a server-integration scheme, are introduced to address this management challenge. The platform-integration scheme integrates and coordinates the management of various coexisting resource managers and associated environments each of which may be managing resources of different types using the most appropriate resource abstraction method. In contrast, the server-integration scheme provides a single, lower level, fine-grained management mechanism across all hardware resource types. Ultimately, the goal of each schemes is to provide a unified view of resources from a capability perspective to consumers.

show abstract

CLOUDLIGHTNING: A Framework for a Self-organising and Self-managing Heterogeneous Cloud

Cited by 32 publications

References 0 publications

Machine learning‐based auto‐tuning for enhanced performance portability of OpenCL applications

Machine learning‐based auto‐tuning for enhanced performance portability of OpenCL applications

Exposing HPC services in the Cloud: the CloudLightning Approach

Heterogeneous Resource Management and Orchestration in Cloud Environments

Contact Info

Product

Resources

About