GENES IV: A bit-serial processing element for a multi-model neural-network accelerator

Abstract

Abstract.A systolic array of dedicated processing elements (PEs) is presented as the heart of a multi-model neural-network accelerator. The instruction set of the PEs makes possible to implement several widely-used neural models, including multi-layer Perceptrons with the back-propagation learning rule and Kohonen feature maps. Each PE holds an element of the synaptic weight matrix. An instantaneous swapping mechanism for the weight matrix makes the efficient implementation of neural networks larger than the physical PE array possible. A systolicallyflowing instruction accompanies each input vector propagating in the array. This avoids the need of emptying and refilling the array when the operating mode of the array is changed. Fixed point arithmetic is used in the PE. The problem of optimally scaling real variables in fixed-point format is addressed.Both the GENES IV chip, containing a matrix of 2 • 2 PEs, and an auxiliary arithmetic circuit have been manufactured and successfully tested. The MANTRA I machine has been built around these chips. Peak performances of the full system are between 200 and 400 MCPS in the evaluation phase and between 100 and 200 MCUPS during the learning phase (depending on the algorithm being implemented).

Mantra I: A Systolic Array for Neural Computation

“…In this section a novel design of the Generic Element for NeuroEmrdator Systolic (GENES) array, called GENES IV [6], is presented.…”

“…A VLSI integrated circuit, named GENES IV [6], has been designed as a building block to implement a square systolic array. It exploits synapse-level parallelism (i.e., one real or virtual processing element (PE) is allocated per synapse or neural connection), while most other systems implement only neuron-level parallelism (i.e., one PE per neuron).…”

On modifications of Kohonen's feature map algorithm for an efficient parallel implementation

“…This is the case for the Kol1oiieii algoritslirii wlieii it has to be irnplementetl on heavily pipelined machines such as the MANTRA I syst~erri built, at, the Swiss Federal Institut,e of Technology [3]. T h e throughput can be increased tflirough bat,cli processing on appropriat,ely designed variants of t,he algoritlim.…”

Implementation of Kohonen's self-organising maps on MANTRA I

“…It is based on a systolic array of up to 40 x 40 Processing Elements dubbed GENES IV ( [2]). The system aims at exploiting the regularity of many ANN algorithms such as Hopfield's networks, the back-propagation rule and Kohonen's Self-Organizing Maps (SOMs).…”