Simplified neuron model as a principal component analyzer

Oja, Erkki

doi:10.1007/bf00275687

Cited by 2,116 publications

(1,396 citation statements)

References 10 publications

Supporting

Mentioning

1,364

Contrasting

Unclassified

Order By: Relevance

“…This statement is interpreted mainly in two different ways. First, a change in the wiring is possible only if both of the connected neurons are active and, thus, correlated (Oja 1982;Bienenstock et al 1982). The second interpretation is that a change should depend only on information that is locally available, that is, the activity of the two neurons and the weight itself (Gerstner and Kistler 2002;Tetzlaff et al 2011).…”

Section: Long-term Plasticitymentioning

confidence: 99%

Time scales of memory, learning, and plasticity

et al. 2012

View full text Add to dashboard Cite

If we stored every bit of input, the storage capacity of our nervous system would be reached after only about 10 days. The nervous system relies on at least two mechanisms that counteract this capacity limit: compression and forgetting. But the latter mechanism needs to know how long an entity should be stored: some memories are relevant only for the next few minutes, some are important even after the passage of several years. Psychology and physiology have found and described many different memory mechanisms, and these mechanisms indeed use different time scales. In this prospect we review these mechanisms with respect to their time scale and propose relations between mechanisms in learning and memory and their underlying physiological basis.

show abstract

Section: Long-term Plasticitymentioning

confidence: 99%

Time scales of memory, learning, and plasticity

et al. 2012

View full text Add to dashboard Cite

show abstract

“…The network settles in two phases, an expectation (minus) phase where the network's actual output is produced, and an outcome (plus) phase where the target output is experienced, and then computes a simple difference of a pre and postsynaptic activation product across these two phases. For Hebbian learning, Leabra uses essentially the same learning rule used in competitive learning or mixtures-of-Gaussians which can be seen as a variant of the Oja normalization (Oja, 1982). The error-driven and Hebbian learning components are combined additively at each connection to produce a net weight change.…”

Section: A4 Hebbian and Error-driven Learningmentioning

confidence: 99%

Attentional control of associative learning—A possible role of the central cholinergic system

Pauli

O’Reilly

2008

Brain Research

View full text Add to dashboard Cite

How does attention interact with learning? Kruschke [Kruschke, J.K. (2001). Toward a unified Model of Attention in Associative Learning. J. Math. Psychol. 45, proposed a model (EXIT) that captures Mackintosh's [Mackintosh, N.J. (1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82(4), 276-298.] framework for attentional modulation of associative learning. We developed a computational model that showed analogous interactions between selective attention and associative learning, but is significantly simplified and, in contrast to EXIT, is motivated by neurophysiological findings. Competition among input representations in the internal representation layer, which increases the contrast between stimuli, is critical for simulating these interactions in human behavior. Furthermore, this competition is modulated in a way that might be consistent with the phasic activation of the central cholinergic system, which modulates activity in sensory cortices. Specifically, phasic increases in acetylcholine can cause increased excitability of both pyramidal excitatory neurons in cortical layers II/III and cortical GABAergic inhibitory interneurons targeting the same pyramidal neurons. These effects result in increased attentional contrast in our model. This model thus represents an initial attempt to link human attentional learning data with underlying neural substrates. KeywordsSelective attention; Associative learning; Layer gain; Acetylcholine; Neural network Except for moments of a "confused, dazed, scatterbrained state which in French is called distraction, and Zerstreutheit in German" (James, 1890, p. 404), we can hardly attend to more than one object or idea at one point in time. Mechanisms of selective attention control which information influences our behavior and our decisions. The focus of our attention can be redirected endogenously because of internal states, or shifted exogenously because of an innate or acquired salience of sensory stimuli. Endogenous attentional control depends on executive functions of the prefrontal cortex (and in the case of spatial attention, interactions with spatial systems in parietal lobes), which enables us to focus our attention based on our current needs or interests (Corbetta and Shulman, 2002). In contrast, this study addresses the neural mechanisms that drive the acquisition of attentional salience for sensory stimuli, which can then overcome the dominance of executive processes to allow our attention to be directed by significant events in our sensory environment. We developed a computational model of learned attentional salience, intended to capture in a very simplified framework the essential dynamics of processing in sensory and association cortex. The central cholinergic system can play an important additional modulatory role as part of a "circuit breaker" (Corbetta and Shulman, 2002) of the fronto-parietal attention system (see also Kimura et al., 1999). Our model incorporates another potential idea a...

show abstract

“…The function f is introduced to keep the weights w~ finite and has to be chosen in a way so that rule (6) extracts the principal component of G. Examples are given by Oja (1982) as f(w) = wrGw and by Yuille et al (1989) as f(w) = wrw, where w r is the transpose of the vector w. The result of a training of the network with these learning rules is an arbitrary weight vector in the subspace spanned by the eigenvectors to 468 the maximal eigenvalue of the correlation matrix G. The behaviour of this model system can therefore be analyzed by simply calculating the principal component vector (vectors) of G for different parameter sets which are introduced in the next subsection.…”

Section: Dynamics and Learnandg Rulementioning

confidence: 99%

“…Again there are no restrictions to the receptive field size and in this model a modified Oja's learning rule was used (Oja 1982) which was shown to lead to cells performing principal component analysis to their input stimuli (which is not the case for Linsker's rule).…”

Section: Introductionmentioning

confidence: 99%

Emergence of orientation selective simple cells simulated in deterministic and stochastic neural networks

1993

View full text Add to dashboard Cite

Abstract. The processing of visual data in area 17 of the mammalian cortex is mainly performed by cells with receptive fields which are tuned to different orientations of input stimuli. The mechanisms underlying the emergence of receptive field properties of orientation selective cells are not well understood up to now. Recently, some models for the prenatal development of the receptive fields of orientation selective simple cells have been proposed, which emerge in neural networks trained by Hebb type unsupervised learning rules. These models, however, use different network architectures and are restricted to the case of identical input neurons. In this work, a biologically motivated neural network model with a general architecture is presented. It is trained with a Hebb type updating rule and with uncorrelated input. The input neurons are identified with retinal ganglion cells and exhibit mature Mexican hat type receptive fields. If the receptive fields of the input neurons have identical properties (deterministic model), a set of parameter domains is found, which characterize different kinds of receptive field maturation behaviour of the network. Results obtained by other authors with similar models are contained in this description as special cases. In addition, the more general and rarely investigated stochastic model, where random variations of the parameters describing the receptive fields of the input neurons occur, is investigated. A high sensitivity of the network against these random variations is obtained. In case of large variations of receptive field parameters of the ganglion cells, a qualitatively new kind of maturation behaviour appears. A significant part of the synaptic connections from ganglion cells to the cortical cell is removed and small simple cell receptive fields with only few lobes emerge. The stochastic model is found to provide a better description of the size, scatter and structure of receptive fields present in biological systems, than the deterministic model.

show abstract

Simplified neuron model as a principal component analyzer

Cited by 2,116 publications

References 10 publications

Time scales of memory, learning, and plasticity

Time scales of memory, learning, and plasticity

Attentional control of associative learning—A possible role of the central cholinergic system

Emergence of orientation selective simple cells simulated in deterministic and stochastic neural networks

Contact Info

Product

Resources

About