Enabling the Adoption of Processing-in-Memory: Challenges, Mechanisms, Future Research Directions

Abstract

Performance improvements from DRAM technology scaling have been lagging behind the improvements from logic technology scaling for many years. As application demand for main memory continues to grow, DRAM-based main memory is increasingly becoming a larger system bottleneck in terms of both performance and energy consumption. A major reason for poor memory performance and energy efficiency is memory's inability to perform computation. Instead, data stored within DRAM memory must be moved into the CPU before any computation can take place. This data movement is costly, as it requires a high latency and consumes significant energy to transfer the data across the pin-limited memory channel. Moreover, the data moved to the CPU is often not reused, and thus does not benefit from being cached within the CPU, which makes it difficult to amortize the overhead of data movement.Modern 3D-stacked DRAM architectures provide an opportunity to avoid unnecessary data movement between memory and the CPU. These multi-layer architectures include a logic layer, where compute logic can be integrated underneath multiple layers of DRAM cell arrays (i.e., the memory layers) within the same chip. Architects can take advantage of the logic layer to perform processing-in-memory (PIM), or near-data processing, where some of the computation is moved from the CPU to the logic layer underneath the memory layer. In a PIM architecture, the logic layer within DRAM has access to the high internal bandwidth available within 3D-stacked DRAM (which is much greater than the bandwidth available in the narrow memory channel between DRAM and the CPU). Thus, PIM architectures can effectively free up valuable bandwidth on the bandwidth-limited memory channel while at the same time reducing system energy consumption.A number of important issues arise when we add compute logic to DRAM. In particular, logic within DRAM does not have low-latency access to common CPU structures that are essential for modern application execution, such as the virtual memory mechanisms, e.g., the translation lookaside buffer (TLB) or the page table walker, and the cache coherence mechanisms, e.g., the coherence directory. To ease the widespread adoption of PIM, we ideally would like to maintain traditional virtual memory abstractions and the shared memory programming model. This requires efficient mechanisms that can provide logic in DRAM with access to virtual memory and cache coherence without having to communicate frequently with the CPU, as off-chip communication between the CPU and DRAM consumes much of the limited bandwidth that PIM aims to avoid using. To this end, we propose and evaluate two general-purpose solutions that can be used by PIM architectures to minimize unnecessary off-chip communication. The first, IMPICA, is an efficient in-memory accelerator for pointer chasing, which can handle address translation entirely within DRAM. The second, LazyPIM, provides coherence support without the need to continually communicate with the CPU. We show that both of these mecha...

QUAC-TRNG: High-Throughput True Random Number Generation Using Quadruple Row Activation in Commodity DRAM Chips

“…Synergy With PIM. Processing-in-memory (PIM) systems improve system performance and/or energy consumption by performing computations directly within a memory chip, thereby avoiding unnecessary data movement [25,26,57,58,60,116,118,137,139]. Prior works propose a broad range of PIM systems [5-8, 13, 22-24, 34, 38, 44, 48, 49, 54, 55, 58, 59, 65, 66, 71, 72, 89, 98, 100, 103, 107, 113, 115, 119, 120, 124, 133-135, 137-139, 142, 148, 164, 168] in the context of various workloads and memory devices.…”

Enabling Practical Processing in and near Memory for Data-Intensive Computing

“…Pushing computation from the CPU into memory introduces new challenges for system architects and programmers to overcome. Many of these challenges must be addressed for PIM to be adopted in a wide variety of systems of workloads, without placing a heavy burden on most programmers [22,73] These challenges include 1) how to easily program PIM systems (with good programming model, library, compiler and tools support) [4,32]; 2) how to design runtime systems and system software that can take advantage of PIM (e.g., runtime scheduling of code on PIM logic, data mapping) [4,9,32,80]; 3) how to efficiently enable coherence between PIM logic and CPU/accelerator cores that operate on shared data [4,10,11]; 4) how to efficiently enable virtual memory support on the PIM logic [33]; 5) how to design high-performance data structures for PIM whose performance is better than concurrent data structures on multi-core machines [65]; 6) how to accurately assess the benefits and shortcomings of PIM using realistic workload suites, rigorous analysis methodologies, and accurate and flexible simulation infrastructures [50,86].…”

“…In this paper, we explore two new approaches to enabling PIM in modern systems. The first approach only minimally changes memory chips to perform simple yet powerful common operations that the chip is inherently efficient at performing [12,13,15,22,23,60,73,[87][88][89][90][91][92][93]. Such solutions take advantage of the existing memory design to perform bulk operations (i.e., operations on an entire row of DRAM cells), such as bulk copy, data initialization, and bitwise operations [13,[88][89][90][91].…”

Processing Data Where It Makes Sense: Enabling In-Memory Computation

“…Despite the extensive design space that we have studied so far, a number of key challenges remain to enable the widespread adoption of PIM in future computing systems [94,95]. Important challenges include developing easy-to-use programming models for PIM (e.g., PIM application interfaces, compilers and libraries designed to abstract away PIM architecture details from programmers), and extensive runtime support for PIM (e.g., scheduling PIM operations, sharing PIM logic among CPU threads, cache coherence, virtual memory support).…”

“…In this paper, we explore two approaches to enabling processing-in-memory in modern systems. The first approach examines a form of PIM that only minimally changes memory chips to perform simple yet powerful common operations that the chip could be made inherently very good at performing [31,71,82,83,84,85,86,90,92,93,94,95,96]. Solutions that fall under this approach take advantage of the existing DRAM design to cleverly and efficiently perform bulk operations (i.e., operations on an entire row of DRAM cells), such as bulk copy, data initialization, and bitwise operations.…”