SNRA: A Spintronic Neuromorphic Reconfigurable Array for In-Circuit Training and Evaluation of Deep Belief Networks

Zand, Ramin; DeMara, Ronald F.

doi:10.1109/icrc.2018.8638604

Cited by 4 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While there has been significant progress in advancing customized silicon DNN hardware (ASICs and FPGAs) [5,9] to improve computational throughput, scalability, and efficiency, their performance (speed and energy efficiency) are fundamentally limited by the underlying electronic components. Even with the recent progress of integrated analog signal processors in accelerating DNNs systems which focus on accelerating matrix multiplication, such as Vector Matrix Multiplying module (VMM) [20], mixed-mode Multiplying-Accumulating unit (MAC) [1,12,27], resistive random access memory (RRAM) based MAC [2,7,8,26,28], etc., the parallelization are still highly limited.…”

Section: Introductionmentioning

confidence: 99%

Real-time Multi-Task Diffractive Deep Neural Networks via Hardware-Software Co-design

Li¹,

Chen²,

Rodriguez³

et al. 2020

Preprint

View full text Add to dashboard Cite

Deep neural networks (DNNs) have substantial computational requirements, which greatly limit their performance in resource-constrained environments. Recently, there are increasing efforts on optical neural networks and optical computing based DNNs hardware, which bring significant advantages for deep learning systems in terms of their power efficiency, parallelism and computational speed. Among them, free-space diffractive deep neural networks (D 2 NNs) based on the light diffraction, feature millions of neurons in each layer interconnected with neurons in neighboring layers. However, due to the challenge of implementing reconfigurability, deploying different DNNs algorithms requires re-building and duplicating the physical diffractive systems, which significantly degrades the hardware efficiency in practical application scenarios. Thus, this work proposes a novel hardware-software co-design method that enables robust and noise-resilient Multi-task Learning in D 2 2NNs. Our experimental results demonstrate significant improvements in versatility and hardware efficiency, and also demonstrate the robustness of proposed multi-task D 2 NN architecture under wide noise ranges of all system components. In addition, we propose a domain-specific regularization algorithm for training the proposed multi-task architecture, which can be used to flexibly adjust the desired performance for each task.

show abstract

Section: Introductionmentioning

confidence: 99%

Real-time Multi-Task Diffractive Deep Neural Networks via Hardware-Software Co-design

Li¹,

Chen²,

Rodriguez³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Nevertheless, matrix multiplication is just one side of the coin in stochastic neural networks. Stochastic sampling forms the other vital operation aligning with the stochastic computing architecture inherent to generative neural networks. ,,− …”

mentioning

confidence: 99%

Restricted Boltzmann Machines Implemented by Spin–Orbit Torque Magnetic Tunnel Junctions

Li,

Wan,

Zhang

et al. 2024

Nano Lett.

View full text Add to dashboard Cite

Artificial intelligence has surged forward with the advent of generative models, which rely heavily on stochastic computing architectures enhanced by true random number generators with adjustable sampling probabilities. In this study, we develop spin−orbit torque magnetic tunnel junctions (SOT-MTJs), investigating their sigmoid-style switching probability as a function of the driving voltage. This feature proves to be ideally suited for stochastic computing algorithms such as the restricted Boltzmann machines (RBM) prevalent in pretraining processes. We exploit SOT-MTJs as both stochastic samplers and network nodes for RBMs, enabling the implementation of RBM-based neural networks to achieve recognition tasks for both handwritten and spoken digits. Moreover, we further harness the weights derived from the preceding image and speech training processes to facilitate cross-modal learning from speech to image generation. Our results clearly demonstrate that these SOT-MTJs are promising candidates for the development of hardware accelerators tailored for Boltzmann neural networks and other stochastic computing architectures.

show abstract

Real-time multi-task diffractive deep neural networks via hardware-software co-design

Chen

Sensale‐Rodriguez

et al. 2021

Sci Rep

View full text Add to dashboard Cite

Deep neural networks (DNNs) have substantial computational requirements, which greatly limit their performance in resource-constrained environments. Recently, there are increasing efforts on optical neural networks and optical computing based DNNs hardware, which bring significant advantages for deep learning systems in terms of their power efficiency, parallelism and computational speed. Among them, free-space diffractive deep neural networks (D2NNs) based on the light diffraction, feature millions of neurons in each layer interconnected with neurons in neighboring layers. However, due to the challenge of implementing reconfigurability, deploying different DNNs algorithms requires re-building and duplicating the physical diffractive systems, which significantly degrades the hardware efficiency in practical application scenarios. Thus, this work proposes a novel hardware-software co-design method that enables first-of-its-like real-time multi-task learning in D22NNs that automatically recognizes which task is being deployed in real-time. Our experimental results demonstrate significant improvements in versatility, hardware efficiency, and also demonstrate and quantify the robustness of proposed multi-task D2NN architecture under wide noise ranges of all system components. In addition, we propose a domain-specific regularization algorithm for training the proposed multi-task architecture, which can be used to flexibly adjust the desired performance for each task.

show abstract

SNRA: A Spintronic Neuromorphic Reconfigurable Array for In-Circuit Training and Evaluation of Deep Belief Networks

Cited by 4 publications

References 33 publications

Real-time Multi-Task Diffractive Deep Neural Networks via Hardware-Software Co-design

Real-time Multi-Task Diffractive Deep Neural Networks via Hardware-Software Co-design

Restricted Boltzmann Machines Implemented by Spin–Orbit Torque Magnetic Tunnel Junctions

Real-time multi-task diffractive deep neural networks via hardware-software co-design

Contact Info

Product

Resources

About