Algorithmic skeletons for multi-core, multi-GPU systems and clusters

Ernsting, Steffen; Kuchen, Herbert

doi:10.1504/ijhpcn.2012.046370

Cited by 66 publications

(53 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are other approaches to simplify GPU programming. SkePU [8] and Muesli [9] are fairly similar to SkelCL, but greatly differ in their focus and implementation, as discussed in [21]. There exist wrappers for OpenCL or CUDA as well as convenient libraries for GPU Computing, most popular of them are Thrust [12] and Bolt [3].…”

Section: Conclusion and Related Workmentioning

confidence: 99%

Towards High-Level Programming for Systems with Many Cores

Gorlatch

Steuwer

2015

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Application development for modern high-performance systems with many cores, i.e., comprising multiple Graphics Processing Units (GPUs) and multi-core CPUs, currently exploits low-level programming approaches like CUDA and OpenCL, which leads to complex, lengthy and error-prone programs. In this paper, we advocate a high-level programming approach for such systems, which relies on the following two main principles: a) the model is based on the current OpenCL standard, such that programs remain portable across various many-core systems, independently of the vendor, and all low-level code optimizations can be applied; b) the model extends OpenCL with three high-level features which simplify many-core programming and are automatically translated by the system into OpenCL code. The high-level features of our programming model are as follows: 1) memory management is simplified and automated using parallel container data types (vectors and matrices); 2) a data (re)distribution mechanism supports data partitioning and generates automatic data movements between multiple GPUs; 3) computations are precisely and concisely expressed using parallel algorithmic patterns (skeletons). The well-defined skeletons allow for semantics-preserving transformations of SkelCL programs which can be applied in the process of program development, as well as in the compilation and optimization phase. We demonstrate how our programming model and its implementation are used to express several parallel applications, and we report first experimental results on evaluating our approach in terms of program size and target performance.

show abstract

Section: Conclusion and Related Workmentioning

confidence: 99%

Towards High-Level Programming for Systems with Many Cores

Gorlatch

Steuwer

2015

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…However, it does not support concurrent execution on CPU and GPU devices nor an intelligent selection mechanism for using CPU or GPU devices. The Muesli skeleton library [90] supports execution on both single-and multi-node MPI-based clusters using OpenMP, MPI and CUDA. However, it does not have an automated mechanism to decide which implementation (OpenMP, CUDA) to use for a given execution context nor does it support concurrent execution on CPU and GPU compute devices.…”

Section: Skepu Librarymentioning

confidence: 99%

Performance-aware component composition for GPU-based systems

Dastgeer¹

2014

View full text Add to dashboard Cite

This thesis adresses issues associated with efficiently programming modern heterogeneous GPU-based systems, containing multicore CPUs and one or more programmable Graphics Processing Units (GPUs). We use ideas from component-based programming to address programming, performance and portability issues of these heterogeneous systems. Specifically, we present three approaches that all use the idea of having multiple implementations for each computation; performance is achieved/retained either a) by selecting a suitable implementation for each computation on a given platform or b) by dividing the computation work across different implementations running on CPU and GPU devices in parallel.In the first approach, we work on a skeleton programming library (SkePU) that provides high-level abstraction while making intelligent implementation selection decisions underneath either before or during the actual program execution. In the second approach, we develop a composition tool that parses extra information (metadata) from XML files, makes certain decisions offline, and, in the end, generates code for making the final decisions at runtime. The third approach is a framework that uses source-code annotations and program analysis to generate code for the runtime library to make the selection decision at runtime. With a generic performance modeling API alongside program analysis capabilities, it supports online tuning as well as complex program transformations.These approaches differ in terms of genericity, intrusiveness, capabilities and knowledge about the program source-code; however, they all demonstrate usefulness of component programming techniques for programming GPU-based systems. With experimental evaluation, we demonstrate how all three approaches, although different in their own way, provide good performance on different GPU-based systems for a variety of applications.This work has been supported by two EU FP7 projects (PEP-PHER, EXCESS) and by SeRC. Populärvetenskaplig sammanfattningAtt få varje generation av datorer att fungera snabbareär viktigt för samhä-llets utveckling och tillväxt. Traditionellt hade de flesta datorer bara en general-purpose processor (den så kallade CPU:n) som bara kunde exekvera en beräkningsuppgift i taget. Under det senasteårtiondet har dock flerkärniga och mångkärniga processorer blivit vanliga, och datorer har också blivit mer heterogena. Ett modernt datorsystem innehåller vanligtvis flerän en CPU, tillsammans med specialprocessorer såsom grafikprocessorer (GPU:er) som ar anpassade för att kunna exekvera vissa typer av beräkningar effektivarë an CPU:er. Vi kallar ett sådant system med en eller flera GPU:er för ett GPU-baserat system. GPU:er i sådana system har sitt eget separata minne, och för att kunna köra en beräkning på en GPU så behöver man vanligtvis flytta all indata till GPU:ns minne och sedan hämta tillbaka resultatet när beräkningenär klar.Programmeringen av GPU-baserade systemär icke-trivialt av flera anledningar: (1) CPU:er och GPU:er kräver olika programmeringsexper...

show abstract

“…The SkelCL library is available as open source software from http://skelcl.uni-muenster.de. Projects like SkePU [4] and Muesli [5] are skeleton-based approaches similar to SkelCL, but differing in focus and implementation, see [13] for comparison. There exist wrappers for OpenCL or CUDA and libraries for GPU Computing, most popular: Thrust [7] and Bolt [2].…”

Section: Conclusion and Related Workmentioning

confidence: 99%

SkelCL: a high-level extension of OpenCL for multi-GPU systems

Steuwer

Gorlatch

2014

J Supercomput

View full text Add to dashboard Cite

Application development for modern high-performance systems with Graphics Processing Units (GPUs) currently relies on low-level programming approaches like CUDA and OpenCL, which leads to complex, lengthy and error-prone programs. We present SkelCL -a high-level programming approach for systems with multiple GPUs and its implementation as a library on top of OpenCL. SkelCL makes three main enhancements to the OpenCL standard: 1) memory management is simplified using parallel container data types (vectors and matrices); 2) an automatic data (re)distribution mechanism allows for implicit data movements between GPUs and ensures scalability when using multiple GPUs; 3) computations are conveniently expressed using parallel algorithmic patterns (skeletons). We demonstrate how SkelCL is used to implement parallel applications and we report experimental evaluation of our approach in terms of programming effort and performance.

show abstract

Algorithmic skeletons for multi-core, multi-GPU systems and clusters

Cited by 66 publications

References 14 publications

Towards High-Level Programming for Systems with Many Cores

Towards High-Level Programming for Systems with Many Cores

Performance-aware component composition for GPU-based systems

SkelCL: a high-level extension of OpenCL for multi-GPU systems

Contact Info

Product

Resources

About