Making Better Use of Processing-in-Memory Through Potential-Based Task Offloading

Abstract

There is an increasing demand for a novel computing structure for data-intensive applications such as artificial intelligence and virtual reality. The processing-in-memory (PIM) is a promising alternative to reduce the overhead caused by data movement. Many studies have been conducted on the utilization of the PIM taking advantage of the bandwidth increased by the through silicon via (TSV). One approach is to design an optimized PIM architecture for a specific application, the other is to find the tasks that will be more advantageous when offloading to PIM. The goal of this paper is to make the PIM, a newly introduced technology, be easily applied to various applications. The programmable GPU-based PIM is the target system. The essential but simple task offloading conditions are proposed to secure as many candidate tasks as possible when there is any potential benefit from the PIM. The PIM design options then are explored reflecting the characteristics of the candidate tasks actively. When determining offloading conditions, it is difficult to simultaneously consider three time-energy-power objectives. Thus, the problem is divided into two sub-problems. The first offloading condition is designed based on time-energy constraints, whereas the second offloading condition is modeled to satisfy time-power constraints. During the whole processes, the offloading conditions and the PIM design options are carefully configured in a complementary manner to reduce the tasks that are excluded from the offloading. In the simulation results, the suitability of the modeled two offloading conditions and the proposed PIM design are verified using various benchmarks and then, they are compared with previous works in terms of processing speed and energy. INDEX TERMS High bandwidth memory, near-data-processing, processing-in-memory, task offloading. I. INTRODUCTION Recently, more and more attention is paid to applications that require massive data processing, such as artificial intelligence and virtual reality. The parallelism of the graphic processing unit (GPU) has been used to increase the processing speed for such applications. However, improving efficiency in terms of data movement has not been a big concern [1]. Therefore, there is an increasing demand for a novel computing structure optimized to the execution of data-intensive applications. The near-data-processing (NDP), which puts the processing unit close to the data, is a promising alternative for reducing the overhead caused by data movement. When the processor is near memory, it is defined as a processing-in-memory (PIM). The associate editor coordinating the review of this manuscript and approving it for publication was Cihun-Siyong Gong. An example is the packaging of a processing unit within a memory module or inside a DRAM chip. Recently, the technology of the through silicon via (TSV) enables CPU, GPU or hardware accelerator to be mounted on logic die of 3-dimensional stacked memory. Thanks to this, PIM technology, which vertically stacks DRAM and a logic die with pr...

A Survey of Resource Management for Processing-In-Memory and Near-Memory Processing Architectures

“…For DCC architectures, solutions can be divided into two main categories: (1) PIM systems, which perform computations using special circuitry inside the memory module or by taking advantage of particular aspects of the memory itself, e.g., simultaneous activation of multiple DRAM rows for logical operations [1,[11][12][13][14][15][16][17][18][19][20][21][22][23][24][25]; (2) NMP systems, which perform computations on a PE placed close to the memory module, e.g., CPU or GPU cores placed on the logic layer of 3D-stacked memory [26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42]. For the purposes of this survey, we classify systems that use logic layers in 3D-stacked memories as NMP systems, as these logic layers are essentially computational cores that are near the memory stack (directly underneath it).…”

“…Programmable logic: Programmable logic PEs can include general purpose processor cores such as CPUs [31,40,[70][71][72], GPUs [26,27,38,73,74], and accelerated processing units (APU) [75] that can execute complex workloads. These cores are usually trimmed down (fewer computation units, less complex cache hierarchies without L2/L3 caches, or lower operating frequencies) from their conventional counterparts due to power, area, and thermal constraints.…”

“…Offloading can be performed at different granularities, e.g., instructions (including small groups of instructions) [1,13,16,19,24,25,28,32,37,39,40,42,57,91,92], threads [71], Nvidia's CUDA blocks/warps [27,29], kernels [26], and applications [38,41,73,74]. Instruction-level offloading is often used with a fixed-function accelerator and PIM systems [1,13,16,19,24,25,28,29,32,37,39,42,57,92]. For example, [42] offloads atomic instructions at instruction-level granularity to a fixed-function near-memory graph accelerator.…”

“…The most common optimization knobs in DCCs include selecting offloading workloads for memory, selecting the most suitable PE in/near memory, or the timing of executing selected offloads. To implement the policy, management techniques have employed code annotation [1,13,16,19,24,25,28,31,32,37,40,57,91,95], compiler-based code analysis [27,39,40,70,92,96], and online heuristics [27][28][29]38,71,72,74]. Table 1 classifies prominent works based on these attributes.…”

“…Overall, the locality aware management policy proves to be the best of the three approaches and demonstrates the importance of correctly deciding which instructions to offload based on locality. Another useful metric to consider when making offloading decisions is the expected memory bandwidth-saving from the offloading [38,42,94]. The speedup achieved by a Host-Only, all offloaded (PIM-Only), and low-locality offload (Locality-Aware) approaches in a DRAM NMP system for different application input sizes [28].…”