Generalization of the backpropagation neural network learning algorithm to permit complex weights

Little, Gordon R.; Gustafson, Steven C.; Senn, R.

doi:10.1364/ao.29.001591

Cited by 22 publications

(4 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other studies in the area of complex-valued networks include a complex backpropagation algorithm to train multilayered feed-forward networks. Details of this work can be found in Little et al (1990), among others. Recently we have proposed discrete and continuous versions of a complex associative memory and have made capacity estimates for the discrete model (Chakravarthy and Ghosh 1994).…”

Section: The Complex Neuron Modelmentioning

confidence: 99%

A complex-valued associative memory for storing patterns as oscillatory states

Chakravarthy

Ghosh

1996

Biological Cybernetics

View full text Add to dashboard Cite

A neuron model in which the neuron state is described by a complex number is proposed. A network of these neurons, which can be used as an associative memory, operates in two distinct modes: (i) fixed point mode and (ii) oscillatory mode. Mode selection can be done by varying a continuous mode parameter, ν, between 0 and 1. At one extreme value of ν (= 0), the network has conservative dynamics, and at the other (ν = 1), the dynamics are dissipative and governed by a Lyapunov function. Patterns can be stored and retrieved at any value of ν by, (i) a one-step outer product rule or (ii) adaptive Hebbian learning. In the fixed point mode patterns are stored as fixed points, whereas in the oscillatory mode they are encoded as phase relations among individual oscillations. By virtue of an instability in the oscillatory mode, the retrieval pattern is stable over a finite interval, the stability interval, and the pattern gradually deteriorates with time beyond this interval. However, at certain values of ν sparsely distributed over ν-space the instability disappears. The neurophysiological significance of the instability is briefly discussed. The possibility of physically interpreting dissipativity and conservativity is explored by noting that while conservativity leads to energy savings, dissipativity leads to stability and reliable retrieval.

show abstract

Section: The Complex Neuron Modelmentioning

confidence: 99%

A complex-valued associative memory for storing patterns as oscillatory states

Chakravarthy

Ghosh

1996

Biological Cybernetics

View full text Add to dashboard Cite

show abstract

“…g1 (IV(r, t)I) a1 IV(r, t)I, (5) where a is a positive constant. g1 (IV(r, t)I) a1 IV(r, t)I, (5) where a is a positive constant.…”

Section: Pcmmentioning

confidence: 99%

“…In all-optical neural networks implemented by using coherent lights[1, 2, 3], the states of neurons and the synaptic weights are represented, respectively, by complex optical fields and complex amplitude transmission functions which have both amplitude and phase information [4,5]. In addition, the complex neural fields in such networks have dynamics that is continuous both in time and space.…”

Section: Introductionmentioning

confidence: 99%

<title>Energy function of complex neural fields generated by optical gain and feedback</title>

Takeda

Kishigami

1993

SPIE Proceedings

View full text Add to dashboard Cite

A complex phase-conjugate neural network model with a Hopfield-like energy function has been proposed, and a physical interpretation is given to the model and its dynamics. The results of experiments and computer simulations are presented that demonstrate the behaviors of the complex neural fields predicted by the theory. 1.INTRODUCTIONIn all-optical neural networks implemented by using coherent lights[1, 2, 3], the states of neurons and the synaptic weights are represented, respectively, by complex optical fields and complex amplitude transmission functions which have both amplitude and phase information [4,5]. In addition, the complex neural fields in such networks have dynamics that is continuous both in time and space. Recently, Noest [6] has proposed a complex neuron model called phasor neuron model, and has shown that a Hopfield-like energy function[8] exists when the synapses have Hermitian symmetry (Tmn T,m ) This model, however, is not suitable for analog optical implementation because it does not allow amplitude variations, and, more importantly, because the law of light propagation, the Helmholtz's reciprocity theorem [7] , demands symmetric synaptic weights (Tmn Tnm ) rather than the Hermitian symmetry as required by Noest's model. We propose an alternative model that is continuous in time and space, allows amplitude variations, and has symmetric synaptic weights to satisfy the Helmholtz's reciprocity theorem. We give a physical interpretation to the model, and point out the existence of a Hopfield-like energy function that governs the dynamics of self-oscillating optical fields in a phase conjugate resonator. We presents results of experiments and computer simulations that demonstrate the behaviors of the complex neural fields predicted by the theory.

show abstract

“…Another interesting extension would be to study a network where all the network parameters are complex numbers. Details of this work can be found in [13], and [14] among others. Recently, a complex version of Hopfield's continous model has been proposed by Hirose [7].…”

Section: Introductionmentioning

confidence: 99%

<title>Studies on a network of complex neurons</title>

Chakravarthy

Ghosh

1993

SPIE Proceedings

View full text Add to dashboard Cite

In the last decade, much effort has been directed towards understanding the role of chaos in the brain. Work with rabbits reveals that in the resting state the electrical activity on the surface ofthe olfactory bulb is chaotic. But, when the animal is involved in a recognition task, the activity shifts to a specific pattern corresponding to the odor that is being recognized. Unstable, quasiperiodic behavior can be found in a class of conservative, deterministic physical systems called the Hamiltonian systems. In this paper, we formulate a complex version of HopfIeld's network of real parameters and show that a variation on this model is a conservative system. Conditions under which the complex network can be used as a Content Addressable memory are studied. We also examine the effect of singularities of the complex sigmoid function on the network dynamics. The network exhibits unpredictable behavior at the singularities due to the failure of a uniqueness condition for the solution of the dynamic equations. On incorporating a weight adaptation rule, the structure of the resulting complex network equations is shown to have an interesting similarity with Kosko's Adaptive Bidirectional Associative Memory.3 SOME PROPERTIES OF THE COMPLEX HOPFIELD NETWORK Lemma 1 LeL z = x + ij and g(z) g + igy tanh(z). Then, 1. >Ofor-ir/4

show abstract

Generalization of the backpropagation neural network learning algorithm to permit complex weights

Abstract: The backpropagation neural network learning algorithm is generalized to include complex-valued interconnections for possible optical implementations.

Cited by 22 publications

References 4 publications

A complex-valued associative memory for storing patterns as oscillatory states

A complex-valued associative memory for storing patterns as oscillatory states

<title>Energy function of complex neural fields generated by optical gain and feedback</title>

<title>Studies on a network of complex neurons</title>

Contact Info

Product

Resources

About