Towards a High Level Approach for the Programming of Heterogeneous Clusters

Heterogeneous distributed computing based on high‐level abstractions

“…Thus, both sides of the problem are tackled in this evaluation. In the work of Viñas et al, we already showed that the separate usage of HTA and HPL, which we call for short HTA+HPL, largely improved the programmability of heterogeneous clusters while incurring in negligible overheads. This way, in order to better assess the advantages of the integrated H 2 TA with respect to HTA+HPL, our evaluation takes as baseline MPI+HPL versions, so that the improvement that HPL means with respect to the usage of standard OpenCL bindings is already present in the baseline.…”

“…In addition, since there are several tools that facilitate the use of OpenCL in C++, it would not make sense to implement the HTA OpenCL support from scratch but rather to rely on some of the existing projects. Given the discussion on OpenCL support in C++ in other works, we took as basis the Heterogeneous Programming Library (HPL), from which we take its runtime and some ideas for the notation of our H 2 TA proposal. For this reason, we will first describe the basics of HPL (Section 4.1) as well as the main drawbacks of the separate use of HTAs and HPL in the same application for programming heterogeneous clusters (Section 4.2) already evaluated in the work of Viñas et al Our new integrated proposal will be then discussed in Section 5.…”

“…These HPL containers are associated to the corresponding local tile of the global HTA, and their purpose is to allow to operate on that tile in an accelerator. As explained in the work of Viñas et al, each HPL Array has its host‐side storage in the same memory area as the associated tile, which is obtained by applying the method raw to the local tile, ie, the one with the index MY_ID, which is the global identifier of the process. The example assumes that the initialization algorithm for x is more efficiently run in the accelerator than in the regular CPU, while the opposite happens for the initialization of y.…”

“…For this reason, and since there is no automatic communication between the runtimes of HTA and HPL, when we want to make some operation using hta_y, we have to explicitly request the update of the host‐side copy of hpl_y, which, as we have just explained, is associated to the same memory as the local tile of hta_y. As detailed in the work of the aforementioned author, this is achieved using method data on this HPL Array, indicating the kind of access which will be performed; in this case, only a read access to perform a reduction across the whole HTA in line 12. As a final step, the example assumes that some computation must be performed in the device in all the tiles of hta_y except the last one.…”

HDArray: Parallel Array Interface for Distributed Heterogeneous Devices

“…Viñas et al [31] proposed the hybrid use of Hierarchically Tiled Array (HTA) [3] for globally distributed arrays and Heterogeneous Programming Library (HPL) [30] for accelerators. Both HTA and HPL C++ libraries provide implicit parallelism and communication and hide many low-level details of MPI and OpenCL; however, there exist two different arrays: an HTA and an HPL Array, which programmers need to define and maintain.…”