An In-DRAM Neural Network Processing Engine

Sudarshan, Chirag; Lappas, Jan; Ghaffar, Muhammad Mohsin; Rybalkin, Vladimir; Weis, Christian; Jung, Matthias; Wehn, Norbert

doi:10.1109/iscas.2019.8702458

Cited by 13 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since this publication, applications ranging from genome string processing to database management systems have all found successful implementations in NDP [29,59,63,69,127,128,129], which increasingly motivates the move to a NDPenabled architecture. All the potential shown by such proposals motivated the creation of commercially available NDP architecture, such as the UPMEM's [82] and the Samsungs' NDP-enabled HBM for artificial intelligence 1 .…”

Section: Related Workmentioning

confidence: 99%

Survey on Near-Data Processing: Applications and Architectures

Santos

Moreira

Cordeiro

et al. 2021

JICS

View full text Add to dashboard Cite

One of the main challenges for modern processors is the data transfer between processor and memory. Such data movement implies high latency and high energy consumption. In this context, Near-Data Processing (NDP) proposals have started to gain acceptance as an accelerator device. Such proposals alleviate the memory bottleneck by moving instructions to data whereabouts. The first proposals date back to the 1990s, but it was only in the 2010s that we could observe an increase in papers addressing NDP. It occurred together with the appearance of 3D-stacked chips with logic and memory stacked layers. This survey presents a brief history of these accelerators, focusing on the applications domains migrated to near-data and the proposed architectures. We also introduce a new taxonomy to classify such architectural proposals according to their data distance.

show abstract

Section: Related Workmentioning

confidence: 99%

Survey on Near-Data Processing: Applications and Architectures

Santos

Moreira

Cordeiro

et al. 2021

JICS

View full text Add to dashboard Cite

show abstract

“…For example, Neurocube [54] executes DNNs using a 3D stacked memory. One major stream of work tries to modify the cell array structure to perform massively parallel DNNs operations within DRAM [34], [48], [65], [92], [97], SRAM [12], [13], [51], [103], and emerging memories [19], [20], [29], [42], [46], [66], [91], [95]. However, these technologies all require changing the cell structure itself, and implementing on top of the legacy technologies is complicated (e.g., DDR4/5 protocol).…”

Section: Related Workmentioning

confidence: 99%

“…Compared to the solutions above, GradPIM is a cheaper, easily realizable solution. Solutions such as [34], [48], [65], [92], [97] involve re-designing the DRAM core cell array. While these are better at performance, they are much harder to realize as a product.…”

Section: Related Workmentioning

confidence: 99%

GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descent

Kim

Park

Kim

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we present GradPIM, a processingin-memory architecture which accelerates parameter updates of deep neural networks training. As one of processing-in-memory techniques that could be realized in the near future, we propose an incremental, simple architectural design that does not invade the existing memory protocol. Extending DDR4 SDRAM to utilize bank-group parallelism makes our operation designs in processing-in-memory (PIM) module efficient in terms of hardware cost and performance. Our experimental results show that the proposed architecture can improve the performance of DNN training and greatly reduce memory bandwidth requirement while posing only a minimal amount of overhead to the protocol and DRAM area.

show abstract

“…In addition, various in-DRAM architectures [25], [52] can support binary networks; however, it is still not possible to quantize into the binary level without a significant quality loss when quantizing large DNN models such as the convolution-based ResNet [10], transformer-based BERT language model [1], and NCF recommendation system [29] over large datasets. DRISA [25] adds simple Boolean logic circuits near the bit line and the local sense amp of the DRAM and runs binary CNNs, but with a prohibitive area overhead of 91% of the base LPDDR4 area.…”

Section: Related Work a Near-memory Processing Dnn Acceleratormentioning

confidence: 99%

McDRAM v2: In-Dynamic Random Access Memory Systolic Array Accelerator to Address the Large Model Problem in Deep Neural Networks on the Edge

et al. 2020

View full text Add to dashboard Cite

The energy efficiency of accelerating hundreds of MB-large deep neural networks (DNNs) in a mobile environment is less than that of a server-class big chip accelerator because of the limited power budget, silicon area, and smaller buffer size of static random access memory associated with mobile systems. To address this challenge and provide powerful computing capability for processing large DNN models in power/resource-limited mobile systems, we propose McDRAM v2, which is a novel in-dynamic random access memory (DRAM) systolic array accelerator architecture. McDRAM v2 makes the best use of large in-DRAM bandwidths for accelerating various DNN applications. It can handle large DNN models without off-chip memory accesses, in a fast and efficient manner, by exposing the large DRAM capacity and large in-DRAM bandwidth directly to an input systolic array of a processing element matrix. Additionally, it maximizes data reuse using a systolic multiply-accumulate (MAC) structure. The proposed architecture maximizes the utilization of large-scale MAC units by judiciously exploiting the DRAM's internal bus and buffer structure. An evaluation of large DNN models in the fields of image classification, natural language processing, and recommendation systems shows that it achieves 1.7 times tera operations per second (TOPS), 3.7 times TOPS/watt, and 8.6 times TOPS/mm 2 improvements over a state-of-the-art mobile graphics processing unit accelerator, and 4.1 times better energy efficiency over a state-of-the-art server-class accelerator. Moreover, it incurs a minimal overhead, i.e., a 9.7% increase in area, and uses less than 4.4 W of peak operating power.

show abstract

An In-DRAM Neural Network Processing Engine

Cited by 13 publications

References 9 publications

Survey on Near-Data Processing: Applications and Architectures

Survey on Near-Data Processing: Applications and Architectures

GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descent

McDRAM v2: In-Dynamic Random Access Memory Systolic Array Accelerator to Address the Large Model Problem in Deep Neural Networks on the Edge

Contact Info

Product

Resources

About