Nonlinear Hebbian Learning as a Unifying Principle in Receptive Field Formation

Abstract

The development of sensory receptive fields has been modeled in the past by a variety of models including normative models such as sparse coding or independent component analysis and bottom-up models such as spike-timing dependent plasticity or the Bienenstock-Cooper-Munro model of synaptic plasticity. Here we show that the above variety of approaches can all be unified into a single common principle, namely nonlinear Hebbian learning. When nonlinear Hebbian learning is applied to natural images, receptive field shapes were strongly constrained by the input statistics and preprocessing, but exhibited only modest variation across different choices of nonlinearities in neuron models or synaptic plasticity rules. Neither overcompleteness nor sparse network activity are necessary for the development of localized receptive fields. The analysis of alternative sensory modalities such as auditory models or V2 development lead to the same conclusions. In all examples, receptive fields can be predicted a priori by reformulating an abstract model as nonlinear Hebbian learning. Thus nonlinear Hebbian learning and natural statistics can account for many aspects of receptive field formation across models and sensory modalities.

Sensory cortex is optimised for prediction of future input

“…This includes having greater power (root mean square) near the present, with brief excitation followed by longer lagging inhibition, producing an asymmetric power profile. This stands in contrast to previous attempts to model RFs based on efficient and sparse coding hypotheses, which either did not capture the diversity of RFs 11 , or lacked temporal asymmetry, punctate structure, or appropriate time scale 9,10,12,13,22,23 .…”

“…We would expect such a model to perform less well in the temporal domain, because unlike the temporal prediction model, the direction of time is not explicitly accounted for. The sparse coding model was chosen because it has set the standard for normative models of visual RFs [4][5][6]19 , and the same model has also been applied for auditory RFs 10,23,27,28 . Past studies 4,5,10 have largely analysed the basis functions produced by the sparse coding model and compared their properties to neuronal RFs.…”

Hebbian plasticity requires compensatory processes on multiple timescales

“…Apart from a few notable exceptions [17,51–54], the vast majority of models of long-term plasticity assume a one-dimensional synaptic state space which represents the synaptic efficacy or weight w ij of a synapse from neuron j to neuron i [5,8,9,27,55–65]. The evolution w ij is then characterized by the differential equationin which the function G , often called the ‘learning rule’, is a member of an infinite dimensional function space , the space of all possible learning rules.…”

“…At the neuronal level, algorithmic normalization of afferent synaptic weights is a commonly used mechanism to stabilize Hebbian plasticity in network models while simultaneously allowing structure formation [9,10,13,14]. While such rapid and precise scaling at the neuronal level has been criticized as biologically implausible [5], an ‘approximate’ scaling could potentially be achieved through heterosynaptic plasticity at the dendritic level [117].…”

“…Owing to these properties, Hebbian plasticity is widely assumed to be the neural basis of associative long-term memory [2–4]. Moreover, Hebbian learning is thought to be the basis of developmental changes such as receptive field development [5–9]. …”