Emergence of simple-cell receptive field properties by learning a sparse code for natural images

Abstract

The receptive fields of simple cells in mammalian primary visual cortex can be characterized as being spatially localized, oriented and bandpass (selective to structure at different spatial scales), comparable to the basis functions of wavelet transforms. One approach to understanding such response properties of visual neurons has been to consider their relationship to the statistical structure of natural images in terms of efficient coding. Along these lines, a number of studies have attempted to train unsupervised learning algorithms on natural images in the hope of developing receptive fields with similar properties, but none has succeeded in producing a full set that spans the image space and contains all three of the above properties. Here we investigate the proposal that a coding strategy that maximizes sparseness is sufficient to account for these properties. We show that a learning algorithm that attempts to find sparse linear codes for natural scenes will develop a complete family of localized, oriented, bandpass receptive fields, similar to those found in the primary visual cortex. The resulting sparse image code provides a more efficient representation for later stages of processing because it possesses a higher degree of statistical independence among its outputs.

Computational modeling and exploration of contour integration for visual saliency

“…However, a morphed neurons structure is still very similar to its original structure and is similar to other feature detectors that operate on seemingly unrelated features. Such a theory would be in agreement with observations that natural images can be described with a relatively small number of Gabor derived kernels very efficiently (Olshausen and Field 1996). As such one might expect the flora of feature detectors to be somewhat constrained at this level of cortex.…”

“…The research thus far agrees with work to date that suggests that V1 neurons are extremely powerful for extracting data from a scene (Olshausen and Field 1996). Additionally, it also helps to validate hypotheses that suggest neurons in V1 have a high dimensionality for visual processing.…”

Functional integration and inference in the brain

“…This principle has proven extremely powerful in predicting some of the basic receptive field properties of cells involved in early visual processing (e.g. Atick and Redlich, 1990;Olshausen and Field, 1996). This principle represents a formal statement of the common sense notion that neuronal dynamics in sensory systems should reflect, efficiently, what is going on in the environment (Barlow, 1961).…”

“…To minimise redundancy one can either minimise the entropy of the output units or maximise their joint entropy, while ensuring the other is bounded in some way. Olshausen and Field (1996) present a very nice analysis based on sparse coding. Sparse coding minimises redundancy using single units with low entropy.…”

The contribution of active body movement to visual development in evolutionary robots

“…Recent advances in computational neuroscience have shown that relatively simple models of developmental visual systems are capable of developing qualitatively similar properties to those found in the early stages of visual processing in cats and monkeys (Hancock et al, 1992;Field, 1994;Olshausen and Field, 1996;Rao and Ballard, 1999). However, those models often use images from publicly available databases or photographs taken in a natural environment as visual stimuli, and do not allow the system to freely interact with the environment and choose those sensory events.…”