Main Memory Scaling: Challenges and Solution Directions

Abstract

The memory system is a fundamental performance and energy bottleneck in almost all computing systems. Recent system design, application, and technology trends that require more capacity, bandwidth, efficiency, and predictability out of the memory system make it an even more important system bottleneck. At the same time, DRAM technology is experiencing difficult technology scaling challenges that make the maintenance and enhancement of its capacity, energy-efficiency, and reliability significantly more costly with conventional techniques.In this chapter, after describing the demands and challenges faced by the memory system, we examine some promising research and design directions to overcome challenges posed by memory scaling. Specifically, we describe three major solution directions: 1) enabling new DRAM architectures, functions, interfaces, and better integration of the DRAM and the rest of the system (an approach we call system-DRAM co-design), 2) designing a memory system that employs emerging non-volatile memory technologies and takes advantage of multiple different technologies (i.e., hybrid memory systems), 3) providing predictable performance and QoS to applications sharing the memory system (i.e., QoS-aware memory systems). We also briefly describe our ongoing related work in combating scaling challenges of NAND flash memory. IntroductionMain memory is a critical component of all computing systems, employed in server, embedded, desktop, mobile and sensor environments. Memory capacity, energy, cost, performance, and management algorithms must scale as we scale the size of the computing system in order to maintain performance growth and enable new applications. Unfortunately, such scaling has become difficult because recent trends in systems, applications, and technology greatly exacerbate the memory system bottleneck.v

Bit-Exact ECC Recovery (BEER): Determining DRAM On-Die ECC Functions by Exploiting DRAM Data Retention Characteristics

“…Today's emerging data-intensive applications and systems in domains such as AI, cloud, and HPC continue to demand greater memory capacity at an unprecedented scale [173][174][175][176][177]. Unfortunately, technology shrinkage throughout the past two decades has yielded diminishing benefits for chip storage capacity, access latency, and refresh overheads because of the growing costs and overheads of maintaining reliable chip operation at smaller technology node sizes [10,11].…”

“…First, improvements to DRAM access latency and storage capacity are slowing down due to DRAM technology scaling challenges [10,11]. To illustrate this problem, we survey 58 publicly-available DRAM chip datasheets 1 in Section 3.1, broadly sampling chips from across 19 DRAM manufacturers spanning the past five decades.…”

“…Second, many promising approaches to address DRAM scaling challenges in today's chips [16,22,29, rely upon exploiting the benefits of deliberately breaking the separation of concerns. These approaches employ system-memory cooperation [10,27,[111][112][113], demonstrating significant system-level benefits from implementing mechanisms outside the DRAM chip to supplement on-chip solutions built by DRAM producers. Section 4 surveys these proposals categorized by the particular DRAM scaling challenge they tackle: (1) improving memory access latencies (Section 4.1); (2) reducing DRAM refresh overheads (Section 4.2); (3) securely defending against the RowHammer vulnerability (Section 4.3); and (4) addressing worsening memory errors (Section 4.4).…”

Cyber Security in Industrial Control Systems (Ics): A Survey of Rowhammer Vulnerability

“…A study showed that given the number of cores increasing faster than DRAM capacity, the expected memory capacity per core would decrease by 30% every two years (Mutlu & Subramanian, 2015). When scaling the size of the computing system, memory also must be scaled, but this makes the maintenance and enhancement of its capacity, energy efficiency, and reliability significantly costlier than conventional techniques (Mutlu, 2015). RH attacks are about the idea that hardware is not so vulnerable and one can attack hardware by exploiting its vulnerabilities.…”

Integer linear programming model for allocation and migration of data blocks in the STT‐RAM‐based hybrid caches

“…The growth in the required data for complex applications and an increase in the number of cores on a single chip lead to high demands for memory capacity and bandwidth [1, 2]. To mitigate the speed gap between processor and off‐chip memory, it is indispensable to exploit large, multi‐level, on‐chip cache hierarchy [3].…”