Heterogeneous memory architectures: A HW/SW approach for mixing die-stacked and off-package memories

Abstract

Die-stacked DRAM is a technology that will soon be integrated in high-performance systems. Recent studies have focused on hardware caching techniques to make use of the stacked memory, but these approaches require complex changes to the processor and also cannot leverage the stacked memory to increase the system's overall memory capacity. In this work, we explore the challenges of exposing the stacked DRAM as part of the system's physical address space. This non-uniform access memory (NUMA) styled approach greatly simplifies the hardware and increases the physical memory capacity of the system, but pushes the burden of managing the heterogeneous memory architecture (HMA) to the software layers. We first explore simple (and somewhat impractical) schemes to manage the HMA, and then refine the mechanisms to address a variety of hardware and software implementation challenges. In the end, we present an HMA approach with low hardware and software impact that can dynamically tune itself to different application scenarios, achieving performance even better than the (impractical-to-implement) baseline approaches.

Hardware Translation Coherence for Virtualized Systems

“…Similarly, Micron's Hybrid Memory Cube [4,5] and byte-addressable persistent memories [6][7][8][9] are quickly gaining traction. Vendors are combining these high-performance memories with traditional high-capacity and low-cost DRAM, prompting research on heterogeneous memory architectures [2,[9][10][11][12][13][14][15].…”

Energy-efficient heterogeneous memory system for mobile platforms

“…Fig. 3 shows the two logical organizations for heterogeneous memory systems (HMS) composed of fast stacked DRAM (e.g., WideIO2) and slow off-package DRAM (e.g., LPDDR4): (a) hardware-managed cache [9,10] and (b) flat physical memory [7,8,10,11]. The cache organization is software-transparent and does not require any change to the software stack.…”

“…If the bandwidth and latency gap between fast and slow regions is small as in mobile DRAMs, the cost of migration can easily outweigh its benefits. Migration at a coarse granularity (say, DRAM page size) [7,11] has advantage of better exploiting DRAM row buffer locality over fine granularity (say, CPU cache block size) [8]. Since the existing migration heuristics are designed for heterogeneous memory devices with a relatively large performance gap [7,8,9,10,11], they can be suboptimal for a LPDDR4-WideIO2 heterogeneous memory system.…”

“…PLT is a table that maintains mappings for migrated pages. In prior work PLT is organized in a hierarchical manner with a master PLT in DRAM and a small on-die PLT cache [7,8,9,11]. To remove the latency cost for an on-die PLT miss, HMMS uses a small on-die PLT only with no master PLT in DRAM, where a PLT miss implies no migration for the requested page.…”

“…In a server environment, the near (fast) memory may have 8Â higher bandwidth than the far (slow) memory and/or much lower latency [7,8,9,10,11]. In this setup it is a good strategy to migrate hot DRAM pages to near memory whenever possible, to maximize the number of requests it services.…”