Logic-Base Interconnect Design for Near Memory Computing in the Smart Memory Cube

Azarkhish, Erfan; Pfister, Christoph; Rossi, Davide; Loi, Igor; Benini, Luca

doi:10.1109/tvlsi.2016.2570283

Cited by 31 publications

(26 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This configuration is found to achieve reasonable performance and efficiency through several simulations. Total available DRAM is 1GB in 4 stacked dies with DRAM banks of 32MB and a closed-page policy [25]. Low-interleavedaddressing is implemented as the HMC's default addressing scheme [26].…”

Section: Resultsmentioning

confidence: 99%

“…A fully functional and cycle-accurate (CA) RTL model of the NeuroCluster has been modeled in SystemVerilog, with the components adopted and reconfigured from [49]. This model along with a previously developed cycle-accurate model of the SMC [25] allows us to analyze the performance of tiled execution over a single SMC device considering the programming overheads.…”

Section: Resultsmentioning

confidence: 99%

“…Heterogeneous Three-dimensional (3D) integration is helping mitigate the well-known memory-wall problem [25] The Through-silicon-via (TSV) technology is reaching commercial maturity by memory manufacturers [26] [27] to build memory cubes made of vertically stacked thinned memory dies in packages with smaller footprint and power compared with traditional multichip modules, achieving higher capacity. On the other hand, a new opportunity for revisiting near-memory computation to further close the gap between processors and memories has been provided in this new context [20] [21].…”

Section: Introductionmentioning

confidence: 99%

“…We also demonstrate that using a flexible tiling mechanism along with a scalable computation paradigm it is possible to efficiently utilize this platform beyond 90% of its roofline [28] limit, and scale its performance to 955 GFLOPS with a network of four SMCs. We have adopted the cycle-accurate SMC model previously developed in [25] along with the generic software stack provided in [24], and built a Register-Transfer-Level (RTL) model for our DL framework, along with the required software layers. Our main contributions can be summarized as follows: I) Using near memory computation for large-scale acceleration of deep ConvNets with large memory footprints, requiring the use of DRAM; II) Proposing the NeuroStream coprocessors as an alternative to vector-processing, providing a flexible form of parallel execution without the need for fine-grained synchronization; III) Presenting a flexible tiling mechanism and a scalable computation paradigm for ConvNets, achieving more than 90% roofline utilization; IV) A low-cost and energyefficient implementation of this solution based on a standard HMC device, scalable to a network of multiple HMCs.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Neurostream: Scalable and Energy Efficient Deep Learning with Smart Memory Cubes

Azarkhish

Rossi

Loi

et al. 2018

IEEE Trans. Parallel Distrib. Syst.

Self Cite

View full text Add to dashboard Cite

Abstract-High-performance computing systems are moving towards 2.5D and 3D memory hierarchies, based on High Bandwidth Memory (HBM) and Hybrid Memory Cube (HMC) to mitigate the main memory bottlenecks. This trend is also creating new opportunities to revisit near-memory computation. In this paper, we propose a flexible processor-in-memory (PIM) solution for scalable and energy-efficient execution of deep convolutional networks (ConvNets), one of the fastest-growing workloads for servers and high-end embedded systems. Our codesign approach consists of a network of Smart Memory Cubes (modular extensions to the standard HMC) each augmented with a many-core PIM platform called NeuroCluster. NeuroClusters have a modular design based on NeuroStream coprocessors (for Convolution-intensive computations) and general-purpose RISC-V cores. In addition, a DRAM-friendly tiling mechanism and a scalable computation paradigm are presented to efficiently harness this computational capability with a very low programming effort. NeuroCluster occupies only 8% of the total logic-base (LoB) die area in a standard HMC and achieves an average performance of 240 GFLOPS for complete execution of full-featured state-of-the-art (SoA) ConvNets within a power budget of 2.5 W. Overall 11 W is consumed in a single SMC device, with 22.5 GFLOPS/W energy-efficiency which is 3.5X better than the best GPU implementations in similar technologies. The minor increase in system-level power and the negligible area increase make our PIM system a cost-effective and energy efficient solution, easily scalable to 955 GFLOPS with a small network of just four SMCs.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Neurostream: Scalable and Energy Efficient Deep Learning with Smart Memory Cubes

Azarkhish

Rossi

Loi

et al. 2018

IEEE Trans. Parallel Distrib. Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In another work, Gao et al [20] developed hardware and software of a 3D stack memory and neardata processing architecture for in-memory analytics frameworks, including MapReduce, graph processing, and deep neural networks. Azarkhish et al [5] developed Smart Memory Cube and designed a high bandwidth interconnect to serve the bandwidth demand of PIM architecture. Zhang et al [49] explored PIM implemented via 3D die stacking.…”

Section: D Processing-in-memory Architecturesmentioning

confidence: 99%

From Processing-in-Memory to Processing-in-Storage

Kaplan

Yavits

Ginosar

2017

JSFI

View full text Add to dashboard Cite

Near-data in-memory processing research has been gaining momentum in recent years. Typical processing-in-memory architecture places a single or several processing elements next to a volatile memory, enabling processing without transferring data to the host CPU. The increased bandwidth to and from volatile memory leads to performance gain. However processing-in-memory does not alleviate von Neumann bottleneck for big data problems, where datasets are too large to fit in main memory.We present a novel processing-in-storage system based on Resistive Content Addressable Memory (ReCAM). It functions simultaneously as a mass storage and as a massively parallel associative processor. ReCAM processing-in-storage resolves the bandwidth wall by keeping computation inside the storage arrays, without transferring it up the memory hierarchy.We show that ReCAM based processing-in-storage architecture may outperform existing processing-in-memory and accelerator based designs. ReCAM processing-in-storage implementation of Smith-Waterman DNA sequence alignment reaches a speedup of almost five over a GPU cluster. An implementation of in-storage inline data deduplication is presented and shown to achieve orders of magnitude higher throughput than traditional CPU and DRAM based systems.Keywords IntroductionUntil the breakdown of Dennard scaling designers focused on improving performance of a single core by increasing instruction level parallelism. In recent years, as Dennard scaling slowed down but Moore's law endured, the focus has shifted to improving parallelism by increasing the number of cores in multicore processors [16]. However, memory bandwidth does not improve at the same rate, making von Neumann bottleneck one of the main performance limiting factors.Data is typically fetched to CPU's main memory from a non-volatile storage such as hard disks or Flash SSDs. Consequently, storage bandwidth and access time pose a major constraint to performance improvement. The problem worsens in datacenter cloud environment, where datasets are distributed among multiple nodes across the datacenter. In such case, data transfer adds latency and reduces bandwidth even further, lowering the performance upper bound.This challenge has motivated renewed interest in Near-Data Processing (NDP) [7]. The main premise of NDP is shifting computing closer to data. NDP seeks to minimize data movement by computing at the most appropriate location in the memory hierarchy, which can be cache, main memory or persistent storage. With NDP, less data needs to be transferred through levels of hierarchy, thus alleviating the limited bandwidth problem. Placing computing resources at the cache level or in main memory (also known as Processing-in-Memory or PiM) does not address the emerging big data problems, where datasets are too large to fit in main memory.Resistive CAM (ReCAM), a storage device based on emerging resistive materials in the bitcell with a novel non-von Neumann Processing-in-Storage (PRinS) compute paradigm, is proposed in order to mitigate the s...

show abstract

Emerging Memory Structures for VLSI Circuits

Garzón¹,

Yavits²,

Lanuzza³

et al. 2022

Wiley Encyclopedia of Electrical and Electronics Engineering

View full text Add to dashboard Cite

Ever since the emergence of the electrical computer and the Von Neumann model, computer architects have adhered to a well‐structured hierarchy of memory solutions, clearly trading off performance and capacity with cost. The ubiquitous memory technologies, dominated by SRAM, DRAM, Flash, and magnetic hard disks, are each situated in a well‐defined location within this hierarchy. However, as processes have scaled into deep nanometer feature sizes and the demand for larger capacities and bandwidths increases, these traditional options face tough challenges and may be limited in their ability to continue to provide the new requirements. New technologies, such as phase change memory (PCM), magnetic RAM (MRAM), and resistive RAM (RRAM), have been researched and developed over the recent past in an attempt to meet these demands and replace some or all of the traditional technologies. In this article, the primary technologies are overviewed, including those that currently fill the memory hierarchy pyramid, the primary candidates to join or replace them in the near‐term, and a number of newer candidates that may arise as legitimate solutions in the farther term. In addition, an overview of the current state of processing within the emerging memory technologies is provided, as an attempt to break free of the traditional Von Neumann paradigm to overcome the energy and performance bottlenecks in modern systems.

show abstract

Logic-Base Interconnect Design for Near Memory Computing in the Smart Memory Cube

Cited by 31 publications

References 42 publications

Neurostream: Scalable and Energy Efficient Deep Learning with Smart Memory Cubes

Neurostream: Scalable and Energy Efficient Deep Learning with Smart Memory Cubes

From Processing-in-Memory to Processing-in-Storage

Emerging Memory Structures for VLSI Circuits

Contact Info

Product

Resources

About