Understanding Latency Variation in Modern DRAM Chips

Abstract

Long DRAM latency is a critical performance bottleneck in current systems. DRAM access latency is defined by three fundamental operations that take place within the DRAM cell array: (i) activation of a memory row, which opens the row to perform accesses; (ii) precharge, which prepares the cell array for the next memory access; and (iii) restoration of the row, which restores the values of cells in the row that were destroyed due to activation. There is significant latency variation for each of these operations across the cells of a single DRAM chip due to irregularity in the manufacturing process. As a result, some cells are inherently faster to access, while others are inherently slower. Unfortunately, existing systems do not exploit this variation.The goal of this work is to (i) experimentally characterize and understand the latency variation across cells within a DRAM chip for these three fundamental DRAM operations, and (ii) develop new mechanisms that exploit our understanding of the latency variation to reliably improve performance. To this end, we comprehensively characterize 240 DRAM chips from three major vendors, and make several new observations about latency variation within DRAM. We find that (i) there is large latency variation across the cells for each of the three operations; (ii) variation characteristics exhibit significant spatial locality: slower cells are clustered in certain regions of a DRAM chip; and (iii) the three fundamental operations exhibit different reliability characteristics when the latency of each operation is reduced.Based on our observations, we propose Flexible-LatencY DRAM (FLY-DRAM), a mechanism that exploits latency variation across DRAM cells within a DRAM chip to improve system performance. The key idea of FLY-DRAM is to exploit the spatial locality of slower cells within DRAM, and access the faster DRAM regions with reduced latencies for the fundamental operations. Our evaluations show that FLY-DRAM improves the performance of a wide range of applications by 13.3%, 17.6%, and 19.5%, on average, for each of the three different vendors' real DRAM chips, in a simulated 8-core system. We conclude that the experimen-Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. tal characterization and analysis of latency variation within modern DRAM, provided by this work, can lead to new techniques that improve DRAM and system performance.

A Lightweight Method to Evaluate Effect of Approximate Memory with Hardware Performance Monitors

“…Second, the memory access latency today affects application performance to the largest extent ever because it has not improve much for the last 20 years while the performance of CPUs have increased exponentially [1], [2]. For example, our previous work observed that the scalability of a memory-access intensive application is worse on a newer machine than on an old machine due to larger performance gap of the CPU and the memory [11].…”

“…Second, the memory access latency relative to the performance of CPU cores is getting larger and larger. This is because the floating point operations per second of cores have been increasing exponentially for decades, while the memory access latency has been almost the same [1], [2]. As a result, memory subsystems today are the largest concern both in terms of the energy consumption and the performance of large-scale computers.…”

“…Prior works have succeeded to lower the latency and energy consumption of main memory by reducing predefined wait parameters of DRAM internal operations so that they consume fewer amount of energy and take less time [2]- [7]. To further lower the latency and energy consumption, using these techniques more aggressively to achieve approximate memory is a promising direction.…”

“…Approximate memory is a type of approximate computing, which realizes increased efficiency such as reduced energy consumption [8], larger capacity [9], and faster throughput [10] in the cost of computation accuracy. By aggressively applying the existing techniques [2]- [7], we can achieve main memory that does not guarantee data integrity but has lower latency and consumes less energy. It is expected to be useful for HPC applications that use iterative methods so that small numerical errors can be amortized, and for AI and big-data analytics applications whose data contain noises by nature.…”

“…The fact that the activation, restoration, and precharge operations incur large overhead both in terms of energy and latency has motivated many prior works to reduce the overhead with novel device-level techniques. Chang et al [2] give a thorough analysis on how reduced activation latency incurs bit-flips, and propose a flexible-latency DRAM system based on their insight. Wang et al [4] shorten the restoration latency by predicting rows that will be accessed in the near future and applying shorter restoration time than defined for these rows because these rows will be fully charged when they are activated again for a future access.…”