Unison Cache: A Scalable and Effective Die-Stacked DRAM Cache

Abstract

Abstract-Recent research advocates large die-stacked DRAM caches in manycore servers to break the memory latency and bandwidth wall. To realize their full potential, diestacked DRAM caches necessitate low lookup latencies, high hit rates and the efficient use of off-chip bandwidth. Today's stacked DRAM cache designs fall into two categories based on the granularity at which they manage data: block-based and page-based. The state-of-the-art block-based design, called Alloy Cache, colocates a tag with each data block (e.g., 64B) in the stacked DRAM to provide fast access to data in a single DRAM access. However, such a design suffers from low hit rates due to poor temporal locality in the DRAM cache. In contrast, the state-of-the-art page-based design, called Footprint Cache, organizes the DRAM cache at page granularity (e.g., 4KB), but fetches only the blocks that will likely be touched within a page. In doing so, the Footprint Cache achieves high hit rates with moderate on-chip tag storage and reasonable lookup latency. However, multi-gigabyte stacked DRAM caches will soon be practical and needed by server applications, thereby mandating tens of MBs of tag storage even for page-based DRAM caches.We introduce a novel stacked-DRAM cache design, Unison Cache. Similar to Alloy Cache's approach, Unison Cache incorporates the tag metadata directly into the stacked DRAM to enable scalability to arbitrary stacked-DRAM capacities. Then, leveraging the insights from the Footprint Cache design, Unison Cache employs large, page-sized cache allocation units to achieve high hit rates and reduction in tag overheads, while predicting and fetching only the useful blocks within each page to minimize the off-chip traffic. Our evaluation using server workloads and caches of up to 8GB reveals that Unison cache improves performance by 14% compared to Alloy Cache due to its high hit rate, while outperforming the state-of-the art page-based designs that require impractical SRAM-based tags of around 50MB.

A fully associative, tagless DRAM cache

“…To best utilize fast, energy-efficient in-package DRAM without burdening software writers, many researchers propose to use it as a large last-level cache [20,21,22,26,27,29,38]. This is justified by the fact that the in-package DRAM capacity, ranging from hundreds of megabytes to several gigabytes [27], is still not big enough to completely replace the main memory especially for emerging applications with huge memory footprints [16] such as in-memory database [7] and genome assemblies [31].…”

“…To alleviate the problems associated with large tags, pagebased DRAM caches have recently been proposed to cache at a page granularity, typically ranging from 1 to 8 kilobytes [20,21,22]. In addition to reducing the tag overhead, page-based caches have additional benefits of higher hit rate by better exploiting spatial locality and maximum DRAM access efficiency by amortizing row activation cost.…”

“…However, the tag overhead is still significant. For 1GB in-package DRAM, most of the existing page-based DRAM caches either require multi-megabytes of on-die SRAM (i.e., 2MB for [21,22]) or allocate 32-64MB in-package DRAM [20] just for tags. This leads to significant overhead in terms of latency, chip area, and energy consumption, which is likely to increase as the in-package DRAM size continues to scale up.…”

“…Eviction from the victim cache is performed asynchronously to take write-back overhead off from the cache access path by having a small number of free blocks always available. Note that techniques to alleviate the over-fetching problem for page-based caches, such as footprint caching [21], hot/cold page tracking [20,22], are complementary and can augment our work.…”

Toward Multi-Layer Holistic Evaluation of System Designs

“…Our choice was motivated by the multidimensional effects of a 3D architecture, as compared to a 2D design, and by the increased interest around this emerging technology [19], [11] 1 . The 3D stacking technique improves performance, due to the higher bandwidth and lower latency of the on-chip DRAM.…”