Reevaluating the latency claims of 3D stacked memories

Chang, Daniel W.; Byun, Gyung-Su; Kim, Ho-Young; Ahn, Minwook; Ryu, Soojung; Kim, N. S.; Schulte, Michael

doi:10.1109/aspdac.2013.6509675

Cited by 18 publications

(6 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Currently, besides the Intel KNL analyzed in this work, other systems such as Altera Stratix 10 and Micron Hybrid Memory Cube are also equipped with HBM [22], [23], [24]. Chang et al [25] show that the most significant performance benefit of 3D-Stack memory is the largely increased bandwidth with respect to conventional DRAM. However, the same study also shows that latency to HBM is not reduced as expected.…”

Section: Related Workmentioning

confidence: 99%

Exploring the Performance Benefit of Hybrid Memory System on HPC Environments

Gioiosa

Kestor

Cicotti

et al. 2017

2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)

View full text Add to dashboard Cite

Abstract-Hardware accelerators have become a de-facto standard to achieve high performance on current supercomputers and there are indications that this trend will increase in the future. Modern accelerators feature high-bandwidth memory next to the computing cores. For example, the Intel Knights Landing (KNL) processor is equipped with 16 GB of high-bandwidth memory (HBM) that works together with conventional DRAM memory. Theoretically, HBM can provide ∼ 4× higher bandwidth than conventional DRAM. However, many factors impact the effective performance achieved by applications, including the application memory access pattern, the problem size, the threading level and the actual memory configuration. In this paper, we analyze the Intel KNL system and quantify the impact of the most important factors on the application performance by using a set of applications that are representative of scientific and data-analytics workloads. Our results show that applications with regular memory access benefit from MCDRAM, achieving up to 3× performance when compared to the performance obtained using only DRAM. On the contrary, applications with random memory access pattern are latency-bound and may suffer from performance degradation when using only MCDRAM. For those applications, the use of additional hardware threads may help hide latency and achieve higher aggregated bandwidth when using HBM.

show abstract

Section: Related Workmentioning

confidence: 99%

Exploring the Performance Benefit of Hybrid Memory System on HPC Environments

Gioiosa

Kestor

Cicotti

et al. 2017

2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)

View full text Add to dashboard Cite

show abstract

“…Heterogeneous main memory. Many memory technologies [17,28,31,40] have been proposed to build HM. Intel Optane DC persistent memory plus traditional DDR is an example [11,32]; High bandwidth memory (HBM) plus DDR in Intel Knights Landing is another example [17].…”

Section: Related Workmentioning

confidence: 99%

“…Many memory technologies [17,28,31,40] have been proposed to build HM. Intel Optane DC persistent memory plus traditional DDR is an example [11,32]; High bandwidth memory (HBM) plus DDR in Intel Knights Landing is another example [17]. Recent research studies data management on HM using hardware-based approaches [9, 14, 55-57, 66, 72, 75] or OS/software-based approaches [23, 26, 42, 51, 53, 62, 69-71, 74, 76].…”

Section: Related Workmentioning

confidence: 99%

Sentinel: Runtime Data Management on Heterogeneous Main MemorySystems for Deep Learning

Ren,

Luo,

et al. 2019

Preprint

View full text Add to dashboard Cite

Software-managed heterogeneous memory (HM) provides a promising solution to increase memory capacity and cost efficiency. However, to release the performance potential of HM, we face a problem of data management. Given an application with various execution phases and each with possibly distinct working sets, we must move data between memory components of HM to optimize performance. The deep neural network (DNN), as a common workload on data centers, imposes great challenges on data management on HM. This workload often employs a task dataflow execution model, and is featured with a large amount of small data objects and fine-grained operations (tasks). This execution model imposes challenges on memory profiling and efficient data migration.We present Sentinel, a runtime system that automatically optimizes data migration (i.e., data management) on HM to achieve performance similar to that on the fast memoryonly system with a much smaller capacity of fast memory. To achieve this, Sentinel exploits domain knowledge about deep learning to adopt a custom approach for data management. Sentinel leverages workload repeatability to break the dilemma between profiling accuracy and overhead; It enables profiling and data migration at the granularity of data objects (not pages), by controlling memory allocation. This method bridges the semantic gap between operating system and applications. By associating data objects with the DNN topology, Sentinel avoids unnecessary data movement and proactively triggers data movement. Using only 20% of peak memory consumption of DNN models as fast memory size, Sentinel achieves the same or comparable performance (at most 8% performance difference) to that of the fast memory-only system on common DNN models; Sentinel also consistently outperforms a state-of-the-art solution by 18%.

show abstract

“…Indeed, the latest Nvidia and AMD GPUs and Intel's Knight's Landing feature die-stacked DRAM [13], [46], [47]. Due to the significant delays involved in routing the request from the requesting core to the desired DRAM bank, access latencies are comparable to main memory [48]. In fact, Intel's Knight's Landing has a higher access latency to the DRAM cache than to main memory [13].…”

Section: Related Workmentioning

confidence: 99%

Farewell My Shared LLC! A Case for Private Die-Stacked DRAM Caches for Servers

Shahab

Zhu

Margaritov

et al. 2018

2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)

View full text Add to dashboard Cite

The slowdown in technology scaling mandates rethinking of conventional CPU architectures in a quest for higher performance and new capabilities. This work takes a step in this direction by questioning the value of on-chip shared lastlevel caches (LLCs) in server processors and argues for a better alternative. Shared LLCs have a number of limitations, including on-chip area constraints that limit storage capacity, long planar interconnect spans that increase access latency, and contention for the shared cache capacity that hurts performance under workload colocation. To overcome these limitations, we propose a Die-Stacked Private LLC Organization (SILO), which combines conventional on-chip private L1 (and optionally, L2) caches with a per-core private LLC in die-stacked DRAM. By stacking LLC slices directly above each core, SILO avoids long planar wire spans. The use of private caches inherently avoids inter-core cache contention. Last but not the least, engineering the DRAM for latency affords low access delays while still providing over 100MB of capacity per core in today's technology. Evaluation results show that SILO outperforms state-of-the-art conventional cache architectures on a range of scale-out and traditional workloads while delivering strong performance isolation under colocation.

show abstract

Reevaluating the latency claims of 3D stacked memories

Cited by 18 publications

References 17 publications

Exploring the Performance Benefit of Hybrid Memory System on HPC Environments

Exploring the Performance Benefit of Hybrid Memory System on HPC Environments

Sentinel: Runtime Data Management on Heterogeneous Main MemorySystems for Deep Learning

Farewell My Shared LLC! A Case for Private Die-Stacked DRAM Caches for Servers

Contact Info

Product

Resources

About