Weight perturbation: an optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayer networks

Jabri, Marwan A.; Flower, B.

doi:10.1109/72.105429

Cited by 193 publications

(36 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several studies have been performed with optimization routines guiding systematic experiments or computational fluid dynamics (CFD) trials (Kern & Koumoutsakos 2006;Kaya & Tuncer 2007;Roberts et al 2009), using algorithms such as weight perturbation (Jabri & Flower 1992) or CMA-ES (Hansen & Ostermeier 2001). However, these methods generally have far slower convergence when compared to model-based methods, where a theoretical model is used to guide the optimization between experimental trials.…”

Section: Optimization Methodsmentioning

confidence: 99%

Adding in-line motion and model-based optimization offers exceptional force control authority in flapping foils

Izraelevitz

Triantafyllou

2014

J. Fluid Mech.

View full text Add to dashboard Cite

We study experimentally the effect of adding an in-line oscillatory motion to the oscillatory heaving and pitching motion of flapping foils that use a power downstroke. We show that far from being a limitation imposed by the muscular structure of certain animals, in-line motion can be a powerful means to either substantially augment the mean lift, or reduce oscillatory lift and increase thrust; propulsive efficiency can also be increased. We also show that a model-based optimization scheme that is used to drive an iterative sequence of experimental runs provides exceptional ability for flapping foils to tightly vector and keep the force in a desired direction, hence improving performance in locomotion and manoeuvring. Flow visualization results, using particle image velocimetry, establish the connection of distinct wake patterns with flapping modes associated with high lift forces, or modes of high thrust and low lift forces.

show abstract

Section: Optimization Methodsmentioning

confidence: 99%

Adding in-line motion and model-based optimization offers exceptional force control authority in flapping foils

Izraelevitz

Triantafyllou

2014

J. Fluid Mech.

View full text Add to dashboard Cite

show abstract

“…The most general perturbation algorithm is called serial weight perturbation [25], [26]. Weights are updated according to (1) where is the learning rate, is the perturbation magnitude, is the perturbed error, and is the nominal error.…”

Section: Learning Algorithms For Hardware Implementationsmentioning

confidence: 99%

Toward a general-purpose analog VLSI neural network with on-chip learning

Montalvo

Gyurcsik

Paulos

1997

IEEE Trans. Neural Netw.

View full text Add to dashboard Cite

This paper describes elements necessary for a general-purpose low-cost very large scale integration (VLSI) neural network. By choosing a learning algorithm that is tolerant of analog nonidealities, the promise of high-density analog VLSI is realized. A 64-synapse, 8-neuron proof-of-concept chip is described. The synapse, which occupies only 4900 mum(2) in a 2-mum technology, includes a hybrid of nonvolatile and dynamic weight storage that provides fast and accurate learning as well as reliable long-term storage with no refreshing. The architecture is user-configurable in any one-hidden-layer topology. The user-interface is fully microprocessor compatible. Learning is accomplished with minimal external support; the user need only present inputs, targets, and a clock. Learning is fast and reliable. The chip solves four-bit parity in an average of 680 ms and is successful in about 96% of the trials.

show abstract

“…For example, if 00 , , In designing a neural network (NN), we focus on the multi-layer feed-forward NN with training done off-line by smoothing the SS hysteresis function to make it differentiable and adapting the back-propagation method 20 . Other techniques such as the weight perturbation method 21 or the extreme learning machine 22 might also be suitable especially for on-chip training. The key difference with standard neural network design is the hysteresis in the threshold function which makes the overall design more robust, but requires an extra condition during the training: the currents arriving at each neuron have to be above threshold 23 .…”

Section: Fan-outmentioning

confidence: 99%

Spin switches for compact implementation of neuron and synapse

Diep

Sutton

Behin‐Aein

et al. 2014

Applied Physics Letters

View full text Add to dashboard Cite

Abstract:Nanomagnets driven by spin currents provide a natural implementation for a neuron and a synapse: currents allow convenient summation of multiple inputs, while the magnet provides the threshold function. The objective of this paper is to explore the possibility of a hardware neural network (HNN) implementation using a spin switch (SS) as its basic building block. SS is a recently proposed device based on established technology with a transistor-like gain and input-output isolation. This allows neural networks to be constructed with purely passive interconnections without intervening clocks or amplifiers. The weights for the neural network are conveniently adjusted through analog voltages that can be stored in a non-volatile manner in an underlying CMOS layer using a floating gate low dropout voltage regulator. The operation of a multi-layer SS neural network designed for character recognition is demonstrated using a standard simulation model based on coupled Landau-Lifshitz-Gilbert (LLG) equations, one for each magnet in the network.

show abstract

Weight perturbation: an optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayer networks

Cited by 193 publications

References 11 publications

Adding in-line motion and model-based optimization offers exceptional force control authority in flapping foils

Adding in-line motion and model-based optimization offers exceptional force control authority in flapping foils

Toward a general-purpose analog VLSI neural network with on-chip learning

Spin switches for compact implementation of neuron and synapse

Contact Info

Product

Resources

About