VLSI implementation of a neural network memory with several hundreds of neurons

Graf, Hans Peter; Jackel, L. D.; Howard, Richard; Straughn, B.; Denker, John S.; Hubbard, W.; Tennant, D. M.; Schwartz, D.

doi:10.1063/1.36253

Cited by 94 publications

(26 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A majority of implementations rely on analog electronics to provide compact neurons possessing the required computational primitives (usually the summation and nonlinear transformation of signals from other neurons in the net). Analog chips have been fabricated containing 512 neurons and a 512 x 512 fixed-resistor matrix which can be programmed to solve a specific problem very rapidly [4]. However, variable resistors (or weights) are needed for the network to learn from previous experience.…”

Section: Stochasticismmentioning

confidence: 99%

TInMANN: the integer Markovian artificial neural network

Bout

Miller

1989

International Joint Conference on Neural Networks

View full text Add to dashboard Cite

A maseively parallel, all-digital, stochastic digital architecture -TIdiANN-is described which performs competitive and Kohonen types of learning at rates as high as 145,000 training examples per second regardless of network size. Simulations of TInMANN, both with and without its conscience mechanism activated, demonstrate its effectiveness on a number of example problems. StochasticismArtificial neural networks are massively parallel architectures that solve difficult problems via the cooperation of highly interconnected but simple computing elements. Such networks form a different paradigm of computing in the hopes of rapidly solving problems while adapting to a changing environment -qualities that are realized in von Neumnnn computers only with great difficulty. For example, vector quantization of images normally requires specialized programming to detect and encode the meaningful clusters of vectors, but a Kohonen self-organizing map can adaptively form the necessary codebook using a simple, general learning algorithm (10). The speed advantage of this and other neural network approaches, however, cannot be realized without specialized hardware.Unfortunately, the large number of processing elements (which we will refer to as neumru) and their highly interconnected nature makes the construction of neural networks challenging. A majority of implementations rely on analog electronics to provide compact neurons possessing the required computational primitives (usually the summation and nonlinear transformation of signals from other neurons in the net). Analog chips have been fabricated containing 512 neurons and a 512 x 512 fixed-resistor matrix which can be programmed to solve a specific problem very rapidly [4]. However, variable resistors (or weights) are needed for the network to learn from previous experience. Such Hopfield neural networks have been built which store their weights on lowleakage capacitors [7] or in digital storage registers [14], while a Kohonen network is under construction which also employs adjustable analog weights [8]. The increased complexity and size of these adjustable weights greatly reduces the number of neurons which can be placed on a chip. Real-world applications will require many neurons, so finding a method of interconnecting these chips to form larger networks will be of primary concern. However, the large number of analog signals which must pass between chips will exceed the available input /output resources, while the noise and parasitic capacitances on the external I/O lines will distort the operation of the network and possibly lead to erroneous results.The limitations of analog computing have led researchers of neural networks to rely upon digital simulation. Currently available high-speed ALUs excel at the inner product computations often found in neural network algorithms and have been used to construct a general-purpose neural network architecture [15] and a dedicated Kohonen network [5]. The size and 1/0 requirements of the digital logic limit the number of ALUs which...

show abstract

Section: Stochasticismmentioning

confidence: 99%

TInMANN: the integer Markovian artificial neural network

Bout

Miller

1989

International Joint Conference on Neural Networks

View full text Add to dashboard Cite

show abstract

“…1) seemed both feasible and useful. [1][2][3][4][5] In the mid 1980s, neural networks were in the first phase of a renaissance, driven by:…”

Section: Introductionmentioning

confidence: 99%

The Prospects for Analogue Neural VLSI

Murray

Woodburn

1997

Int. J. Neur. Syst.

View full text Add to dashboard Cite

In recent years, the efforts of analogue, neural-hardware designers have shifted from generic analogue neurocomputers to "niche" markets in sensor fusion and robotics, and we explain why this is so. We describe the main differences between digital and analogue computation, and consider the advantages of pure analogue and pulsed methods of design. We then investigate some important issues in analogue design of neural machines, namely weight storage (volatile and non-volatile), on-chip learning, and arithmetic accuracy and its relationship to noise. Finally, we outline those areas in which analogue techniques are likely to prove most useful, and speculate as to their likely long-term utility.

show abstract

“…Analog implementations of the Hopfield network containing up to 512 neurons have been built with matrices of fixed resistors and nonlinear amplifiers fabricated on a single chip [9]. Such a system is able to solve a specific problem very rapidly, but variable resistors are needed in order to change the problem constraints or to allow the network to "learn" from previous experience.…”

Section: Introductionmentioning

confidence: 99%

A stochastic architecture for neural nets

Bout

Miller

1988

IEEE International Conference on Neural Networks

View full text Add to dashboard Cite

A rtochastic digital architecture is described for simulating the operation of Hopfield neural networks. This architecture provider reprogrammability (since synaptic weightr are stored in digital rhift regirterr), large dynamic range (by using either fixed or floating-point weightr), annealing (by coupling variable neuron with noise from stochastic arithmetic), high execution speeds (x N -10' connections per second), expandability (by cascading of multiple chips to host large networks), and practicality (by building with very conservative MOS device technologies).

show abstract

VLSI implementation of a neural network memory with several hundreds of neurons

Cited by 94 publications

References 0 publications

TInMANN: the integer Markovian artificial neural network

TInMANN: the integer Markovian artificial neural network

The Prospects for Analogue Neural VLSI

A stochastic architecture for neural nets

Contact Info

Product

Resources

About