MACC: An OpenACC Transpiler for Automatic Multi-GPU Use

Abstract

Abstract. Graphics Processing Units (GPUs) perform the majority of computations in state-of-the-art supercomputers. Programming these GPUs is often assisted using a programming model such as (amongst others) the directive-driven OpenACC. Unfortunately, OpenACC (and other similar models) are incapable of automatically targeting and distributing work across several GPUs, which decreases productivity and forces needless manual labor upon programmers. We propose a method that enables OpenACC applications to target multi-GPU. Workload distribution, data transfer and inter-GPU communication (including modern GPU-to-GPU links) are automatically and transparently handled by our compiler with no user intervention and no changes to the program code. Our method leverages existing OpenMP and OpenACC backends, ensuring easy integration into existing HPC infrastructure. Empirically we quantify performance gains and losses in our data coherence method compared to similar approaches, and also show that our approach can compete with the performance of hand-written MPI code.

Multi-accelerator extension in OpenMP based on PGAS model

“…Note that our idea may be inferior in terms of productivity compared to [12]. However, the purpose of [12] is to execute a code on multiple accelerators environment without changing code on a single accelerator environment, so the purpose is different from ours. In addition, unlike [10,26], our method does not change the host side programming.…”

“…Also, unlike [23], the proposed directives can upload data at an arbitrary position. Moreover, unlike [12] proposed directives can utilize all accelerator memory effectively. In addition, while [23] and [12] may insert communication implicitly, our proposed OpenMP extension makes users insert communication explicitly using simple directives to perform performance tuning easily.…”

“…Moreover, unlike [12] proposed directives can utilize all accelerator memory effectively. In addition, while [23] and [12] may insert communication implicitly, our proposed OpenMP extension makes users insert communication explicitly using simple directives to perform performance tuning easily. Note that our idea may be inferior in terms of productivity compared to [12].…”

“…In addition, while [23] and [12] may insert communication implicitly, our proposed OpenMP extension makes users insert communication explicitly using simple directives to perform performance tuning easily. Note that our idea may be inferior in terms of productivity compared to [12]. However, the purpose of [12] is to execute a code on multiple accelerators environment without changing code on a single accelerator environment, so the purpose is different from ours.…”

“…Therefore, the OpenACC extension cannot upload data in an arbitrary place. Matsumura et al [12] develop an OpenACC compiler system to generate an OpenACC code for multiple accelerators from an OpenACC code for a single accelerator automatically. However, there are some limitations.…”