Analog Vector-Matrix Multiplier Based on Programmable Current Mirrors for Neural Network Integrated Circuits

A Low-Voltage, Low-Power Reconfigurable Current-Mode Softmax Circuit for Analog Neural Networks

“…The basic shape of the softmax function is preserved also for the smallest technology option, especially in the linear region. However, due to an increased offset as a result of I SCALE being adjusted to match the upper part, a worsened matching in the linear region can be observed, resulting in a higher relative error, with a peak value close to 6.5%, which can be still reasonable since there are simple DNNs which can operate with a reduced equivalent number of bits [3]. Finally, Figure 10 shows the transfer characteristics (a) and related errors (b) of a softmax function simulated with three different technology nodes, namely TSMC 180 nm, 65 nm, and 40 nm, in order to investigate the impact of technology scaling.…”

“…Their highly parallelized and interconnected architecture is not naturally implementable by conventional arithmetic logic units (ALUs) of modern microprocessors. In this context, the possible implementation of DNNs fully or partially realized in the analog domain is attracting a lot of attention [1][2][3][4]. A DNN architecture is generally composed of one input layer, two or more hidden layers, and one output layer.…”

“…When compared to digital implementations, analog designs are faster and more power-efficient [2,11]. Although this is normally achieved at the cost of higher sensitivity to process-voltage-temperature (PVT) variations, it is well known that this issue can be tolerated to some extent due to the inherent resilience of DNNs to variability [3].…”

Single-poly floating-gate memory cell options for analog neural networks

“…The most frequently recurring building block of a DNN is the vectormatrix multiplier (VMM), for which a block diagramwhich can also be interpreted as a parallel architectureis illustrated in The recurring arithmetic operations implemented in a VMM cannot be efficiently performed by a general-purpose CPU, thus various degrees of parallel and/or logic in-memory architectures need to be implemented to boost overall DNN performance and efficiency [2], [3]. In this context, together with digital approaches such as GPU-based hardware and ASIC accelerators [4], computation in the analog domain is gaining momentum as a longer term solution [5]- [10], on the basis of the observation that limited equivalent arithmetic precision in the inference phase is sufficient for a DNN to achieve a high classification accuracy © 2021. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/ The final publication is available at: https://doi.org/10.1016/j.sse.2021.108062 [11].…”

“…Analog architectures can reach a very high computation efficiency, exploiting fundamental circuit laws and intrinsic device properties to execute in-memory arithmetic operations: addition, for example, can be simply obtained by exploiting Kirchhoff's current law in summing several currents coming from various branches and injected into the same node (Figure 1). On the other hand, it is well known that analog processing blocks can be affected by circuit non-idealities such as noise, non-linearity and process variations [5], [9], [10] which can be counteracted at the cost of increased area occupation.…”