Toward Exascale Computing Systems: An Energy Efficient Massive Parallel Computational Model

Abstract

The emerging Exascale supercomputing system expected till 2020 will unravel many scientific mysteries. This extreme computing system will achieve a thousand-fold increase in computing power compared to the current petascale computing system. The forthcoming system will assist system designers and development communities in navigating from traditional homogeneous to the heterogeneous systems that will be incorporated into powerful accelerated GPU devices beside traditional CPUs. For achieving ExaFlops (10 18 calculations per second) performance through the ultrascale and energy-efficient system, the current technologies are facing several challenges. Massive parallelism is one of these challenges, which requires a novel energy-efficient parallel programming (PP) model for providing the massively parallel performance. In the current study, a new parallel programming model has been proposed, which is capable of achieving massively parallel performance through coarse-grained and fine-grained parallelism over internode and intra-node architectural-based processing. The suggested model is a tri-level hybrid of MPI, OpenMP and CUDA that is computable over a heterogeneous system with the collaboration of traditional CPUs and energy-efficient GPU devices. Furthermore, the developed model has been demonstrated by implementing dense matrix multiplication (DMM). The proposed model is considered an initial and leading model for obtaining massively parallel performance in an Exascale computing system.

M2C: A Massive Performance and Energy Throttling Framework for High-Performance Computing Systems

“…To achieve massive parallelism in parallel computing, a Tri-Hierarchy hybrid MOC (MPI + OpenMP + CUDA) model was proposed in 2018 [20]. This model helped in achieving massive performance through monolithic parallelism when we compute any HPC application over a large-scale cluster system having multiple nodes and a number of GPUs > 2.…”

“…The overall performance of HPC system is considered as the most fundamental and essential metric in massive parallel programming which is measured in a total number of achieved floating-point operations per second (Flops) [32]. The total number of achieved Flops (F T ) can be calculated by dividing the achieved Flops calculated at the peak performance of the system (F pp ) by the total execution time (T Exec ) [20], which can be written as follows:…”

“…We have shown the running states of Open Hardware Monitor and GPU-Z.2.6.0 (see Appendix 1 and Appendix 2, respectively). The total energy consumed by a system Esys can be calculated by integrating the energy consumption composed of memory contention, bandwidth, parallelism and behavior of the application [20] as follows:…”

“…The power consumption in a heterogeneous system is considered into three major parts including power consumption by host CPU processors, power consumption by memory operations (inter-memory and intra-memory) and power consumption by accelerated GPU devices. Power consumption for the systems having GPU installed on it is calculated by [20] as follows,…”

“…The power consumption on a specific application (P app ) of the system can be calculated as follows [20],…”