Effect of Program Error in Memristive Neural Network With Weight Quantization

Kim, Tae-Hyeon; Kim, Sung-Joon; Hong, Kyungho; Park, Jin-Woo; Youn, Sangwook; Lee, Jong‐Ho; Park, Byung‐Gook; Kim, Hyungjin

doi:10.1109/ted.2022.3169112

Cited by 23 publications

(14 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A possible hardware realization based on VCM cells of such a dense layer is sketched in Figure 8d, see refs. [68][69][70][71]. For our simulation approach, we use the conductance (=inverse cell resistance) for the representation of the weights of the dense layer.…”

Section: Statistical Influence On Neuronal Network Applicationmentioning

confidence: 99%

A Physical Description of the Variability in Single‐ReRAM Devices and Hardware‐Based Neuronal Networks

Funck,

Wiefels,

Bengel

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

Resistive switching devices are candidates to be used as weight for the future hardware realization of neuronal networks. Herein, these resistive switches base on oxygen vacancy migration and can be toggled between different resistance states. Internal processes lead to a variability over time in these states. To understand this, a dynamical model is developed, which links the internal physical mechanism to the current response. Thereby, the electronic current is calculated by tunneling processes over oxygen vacancies. The model allows to understand the physical transport properties and the current–voltage dependence. Integrating ionic hopping processes clarifies the variability in the electronic conduction. The derived model does not require empirical input parameters and only depends on the properties of the materials. This allows to use literature values and independently compare them to electrical measurements. The model predicts the experiments correctly. Thereupon, the physical properties of the conduction are investigated with respect to the transport over discrete energy levels, which change under the hopping of oxygen vacancies. Afterwards, the current level and variability are simulated for increasing oxygen vacancy concentrations to investigate analog switching, which is required for neuronal network layers. Finally, the variability is tested with multiple interacting cells in a neuronal layer.

show abstract

Section: Statistical Influence On Neuronal Network Applicationmentioning

confidence: 99%

A Physical Description of the Variability in Single‐ReRAM Devices and Hardware‐Based Neuronal Networks

Funck,

Wiefels,

Bengel

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…Simplifications such as binarized neural networks (BNNs) and XNOR neural networks are suited for hardware implementation thanks to the fast operational speed, low computational power requirements, and resilience to hardware-induced noise signals. − Most of these binarized outputs represent “–1” and “1” with a boundary of 0, which may require two wordlines (WLs) or an additional negative voltage to apply the negative signal to the next layer in the hardware system . To compromise this, a binary activation function to “0” and “1” can be used, but the boundary is still 0, which can generate excess output signals more than necessary. − Additionally, low-memory usage networks such as quantized neural networks (QNNs) can be also considered for hardware-based neural networks, as they alleviate the burden of device state control through weight quantization. − …”

Section: Introductionmentioning

confidence: 99%

Implementation of Convolutional Neural Networks in Memristor Crossbar Arrays with Binary Activation and Weight Quantization

Park,

Kim,

Song

et al. 2024

ACS Appl. Mater. Interfaces

Self Cite

View full text Add to dashboard Cite

We propose a hardware-friendly architecture of a convolutional neural network using a 32 × 32 memristor crossbar array having an overshoot suppression layer. The gradual switching characteristics in both set and reset operations enable the implementation of a 3-bit multilevel operation in a whole array that can be utilized as 16 kernels. Moreover, a binary activation function mapped to the read voltage and ground is introduced to evaluate the result of training with a boundary of 0.5 and its estimated gradient. Additionally, we adopt a fixed kernel method, where inputs are sequentially applied to a crossbar array with a differential memristor pair scheme, reducing unused cell waste. The binary activation has robust characteristics against device state variations, and a neuron circuit is experimentally demonstrated on a customized breadboard. Thanks to the analogue switching characteristics of the memristor device, the accurate vector−matrix multiplication (VMM) operations can be experimentally demonstrated by combining sequential inputs and the weights obtained through tuning operations in the crossbar array. In addition, the feature images extracted by VMM during the hardware inference operations on 100 test samples are classified, and the classification performance by off-chip training is compared with the software results. Finally, inference results depending on the tolerance are statistically verified through several tuning cycles.

show abstract

“…[1][2][3][4][5][6] It also showed great potential in implementing the hardware of diverse neuromorphic networks as the synaptic weight-representing device or temporal/ physical kernels. [7][8][9][10] For both applications, the passive configuration of the CBA renders the sneak current a severe problem. Sneak current flows in the plurality of parallel paths where the minimum resistance is formed.…”

Section: Introductionmentioning

confidence: 99%

Graph Analysis with Multifunctional Self‐Rectifying Memristive Crossbar Array

et al. 2023

View full text Add to dashboard Cite

Many big data have interconnected and dynamic graph structures growing over time. Analyzing these graphical data requires the hidden relationship between the nodes in the graphs to be identified, which has conventionally been achieved by finding the effective similarity. However, graphs are generally non‐Euclidean, which does not allow finding it. In this study, the non‐Euclidean graphs are mapped to a specific crossbar array (CBA) composed of self‐rectifying memristors and metal cells at the diagonal positions. The sneak current, an intrinsic physical property in the CBA, allows for the identification of the similarity function. The sneak‐current‐based similarity function indicates the distance between the nodes, which can be used to predict the probability that unconnected nodes will be connected in the future, connectivity between communities, and neural connections in a brain. When all bit lines of the CBA are connected to the ground, the sneak current is suppressed, and the CBA can be used to search for adjacent nodes. This work demonstrates the physical calculation methods applied to various graphical problems using the CBA composed of the self‐rectifying memristor based on the HfO2 switching layer. Moreover, such applications suffer less from the memristors’ inherent issues related to their stochastic nature.

show abstract

Effect of Program Error in Memristive Neural Network With Weight Quantization

Cited by 23 publications

References 38 publications

A Physical Description of the Variability in Single‐ReRAM Devices and Hardware‐Based Neuronal Networks

A Physical Description of the Variability in Single‐ReRAM Devices and Hardware‐Based Neuronal Networks

Implementation of Convolutional Neural Networks in Memristor Crossbar Arrays with Binary Activation and Weight Quantization

Graph Analysis with Multifunctional Self‐Rectifying Memristive Crossbar Array

Contact Info

Product

Resources

About