Time-Domain Weighted-Sum Calculation for Ultimately Low Power VLSI Neural Networks

Abstract

A time-domain analog weighted-sum calculation model is proposed based on an integrate-and-fire-type spiking neuron model. The proposed calculation model is applied to multi-layer feedforward networks, in which weighted summations with positive and negative weights are separately performed in each layer and summation results are then fed into the next layers without their subtraction operation. We also propose very large-scale integrated (VLSI) circuits to implement the proposed model. Unlike the conventional analog voltage or current mode circuits, the time-domain analog circuits use transient operation in charging/discharging processes to capacitors. Since the circuits can be designed without operational amplifiers, they can operate with extremely low power consumption. However, they have to use very high resistance devices on the order of GΩ. We designed a proof-of-concept (PoC) CMOS VLSI chip to verify weighted-sum operation with the same weights and evaluated it by post-layout circuit simulation using 250-nm fabrication technology. High resistance operation was realized by using the subthreshold operation region of MOS transistors. Simulation results showed that energy efficiency for the weighted-sum calculation was 290 TOPS/W, more than one order of magnitude higher than that in state-of-the-art digital AI processors, even though the minimum width of interconnection used in the PoC chip was several times larger than that in such digital processors. If state-of-the-art VLSI technology is used to implement the proposed model, an energy efficiency of more than 1,000 TOPS/W will be possible. For practical applications, development of emerging analog memory devices such as ferroelectric-gate FETs is necessary.

Use of Virtual Forward Propagation Network Model to Translate Analog Components

“…This is due to the overhear associated with coding a back-propagation algorithm on an arbitrary network and is forgone in favor of a randomly generated network. Since a standard activation function is used to predict the transfer function of CMOS inverter, the virtual network could theoretically be trained using a standard back-propagation algorithm [10,13,21,22]. This training would need to be tested for its rate of convergence, but other than there appears to be no reason why it would not be effective as the derivative which is still a linear combination of the output.…”

“…If a programmable version of the forward propagation network could be implemented, then it is possible for some new computer architectures to be explored [3,21,22]. The idea here is that a chip could be included in the CPU which would be programmable.…”

“…These functions not only determine theoretically but also locally. This programmable network would allow the designer an application specific integrated chip that could be included as a part of a software implementation [16,18,21].…”

“…When a brain cell receives enough stimulation from the adjacent brain cells, it will broadcast a signal to its neighboring neurons that will activate other neurons. This system of activating and associate neurons are the fundamental mechanism in what provides the functionality of the human brain [5,6,21,22]. In software, this phenomenon is simulated as an artificial neural network (ANN).…”

“…After combining with a CMOS inverter, the perceptron would serve as an activation function and operational amplifier to buffer the output. Connections between neurons could weight by using resistors in conjunction with wires to span different values [5,7,21].…”