Study of Quantized Hardware Deep Neural Networks Based on Resistive Switching Devices, Conventional versus Convolutional Approaches

Abstract

A comprehensive analysis of two types of artificial neural networks (ANN) is performed to assess the influence of quantization on the synaptic weights. Conventional multilayer-perceptron (MLP) and convolutional neural networks (CNN) have been considered by changing their features in the training and inference contexts, such as number of levels in the quantization process, the number of hidden layers on the network topology, the number of neurons per hidden layer, the image databases, the number of convolutional layers, etc. A reference technology based on 1T1R structures with bipolar memristors including HfO2 dielectrics was employed, accounting for different multilevel schemes and the corresponding conductance quantization algorithms. The accuracy of the image recognition processes was studied in depth. This type of studies are essential prior to hardware implementation of neural networks. The obtained results support the use of CNNs for image domains. This is linked to the role played by convolutional layers at extracting image features and reducing the data complexity. In this case, the number of synaptic weights can be reduced in comparison to MLPs.

Performance Assessment of Amorphous HfO2-Based RRAM Devices for Neuromorphic Applications

“…Moreover, the TEM cross-sectional images that confirm the layer thickness information and also the amorphous state of the HfO 2 film were previously published and can be seen at Ref. 21 and Ref. 19 Each device occupies an area of 75 × 75 μm 2 .…”

Recent Progress in Selector and Self‐Rectifying Devices for Resistive Random‐Access Memory Application

“…The device has been well studied to demonstrate its feasibility for memory and neuromorphic applications. [ 98 ] Liu et al fabricated a 1T1R device to improve the uniformity of the RS process. They fabricated 1T using a 0.13 μm logic process and then integrated it with a Cu/HfO x /Pt RRAM device.…”

“…d) Reproduced under the terms of the CC‐BY Creative Commons Attribution 4.0 International license ( https://creativecommons.org/licenses/by/4.0). [ 98 ] Copyright 2021, The Authors, published by MDPI.…”

Variability Estimation in Resistive Switching Devices, a Numerical and Kinetic Monte Carlo Perspective

“…Another booming application where resistive switching devices are called to play an important role is linked to neuromorphic computing. These devices can mimic biological synapses in order to simplify the fabrication of hardware neural networks to build artificial intelligence accelerators [9][10][11][12][13][14][15][16][17][18][19][20][21][22]. Neuromorphic circuits have greatly evolved since the first designs proposed by Carver Mead [23]; in recent years, crossbar arrays made of resistive switching devices have shown the way to implement vector-matrix multiplication, a key module both for the training and inference functions in hardware neural networks.…”

“…Neuromorphic circuits have greatly evolved since the first designs proposed by Carver Mead [23]; in recent years, crossbar arrays made of resistive switching devices have shown the way to implement vector-matrix multiplication, a key module both for the training and inference functions in hardware neural networks. In addition to the scaling and low power operation possibilities related to resistive switching devices in the neuromorphic landscape, it has been reported how their inherent variability can be used to improve commonly found hurdles in machine learning, such as overfitting [18,24].…”