Preserving properties of object shape by computations in primary visual cortex

Abstract

Although our visual system is extremely good at extracting objects from the visual scene, this process involves complicated computations that are thought to require image processing by many successive cortical areas. Thus, intermediate stages in object extraction should not eliminate essential properties of the objects that are still required by later stages. A particularly important characteristic of an object is its shape, and shape has the property that it is unchanged by translations, rotations, and magnifications of the image. I show that the requirement for this property of shape to be preserved in the image, as represented by the firing of neurons in the primary visual cortex (V1), is equivalent to a particular type of computation, known as a wavelet transform, determining the firing rate of V1 neurons in response to an image on the retina. Experimental data support the conclusion that the neural representation of images in V1 is described by a wavelet transform and, therefore, that the properties of shape are preserved.A well performed mechanical experiment, such as rolling balls down an inclined plane, gives the same result regardless of whether it is done in China or Italy and whether it was done yesterday or centuries ago. The remarkable thing, which was proved by Emmy Noether nearly a century ago, is that this simple observation (i.e., experimental results do not depend on the time or place at which the experiment is done) requires that energy and momentum must be conserved. This result is an example of what is called a ''symmetry argument'' because doing something, such as carrying out an experiment at a different place or time, has no effect on the result, just as rotating a 4-fold symmetric object such as a square by 90°does not change its appearance. Although arguments of this sort are very common in physics (1), they are rarely used in biology. Here, my goal is to use a symmetry argument to gain insights into how the cortex processes information. I identify abstract properties of an image that should be preserved in the cortical representation of that image (a kind of symmetry argument, because abstract image properties are preserved with cortical processing) and explore the consequences of this preservation for the types of calculations that the cortex can perform.We see the world as filled with objects, but extracting them from the complicated pattern of light and dark that is presented to the retina requires complex computations by a sequence of cortical areas. One of the most important defining characteristics of an object is its shape. Because objects are not extracted in the primary visual cortex (V1) but rather in later areas (2), the representation of an image in V1 should not eliminate the essential properties of objects needed for computations by other areas. I show that this requirement for preserving abstract shape properties (described below) in the cortical representation of images restricts the computations that V1 can carry out to a particular class known as wavelet transforms (3...

Visual object categorization in birds and primates: Integrating behavioral, neurobiological, and computational evidence within a “general process” framework

“…The answer lies in the fact that, at the beginning of the hierarchy of processing, the image is filtered using a battery of Gabor functions at several different scales. In this wavelet-like decomposition of the image (Field, 1999;Stevens, 2004), which is carried out by V1 neurons in the primate brain, filters at the largest scales can detect changes in luminance such as those produced by a gray object over a white background; filters at smaller scales can detect changes in luminance such as those produced by the reflection of light on the smooth surface of the object; and filters at the smallest scales can detect local changes in luminance such as those produced by the object's edges. During learning, the model relies on all of this information for recognition.…”

Novel neural circuit mechanism for visual edge detection

“…The receptive field of a V1 cell, in this simple case, is Ψ p ðν; xÞ = e −iνx , where the position of the neuron in the onedimensional cortex is given by ν, and this receptive field function is the one that preserves object properties (11). The response (firing rate) FðνÞ of a V1 neuron is, as usual, the inner product of the receptive field function Ψ p ðν; xÞ with the stimulus SðxÞ,…”

“…As several authors have noted, simple cells in V1 can be described as performing a wavelet transform (11,12). What this means is that an idealized description of the V1 computation conforms to a wavelet transform in which the firing rate of each V1 simple cell gives the weight on one basis function in a wavelet transform of the visual scene and that the range of receptive fields present in V1 is sufficient to provide an accurate representation of the image as presented to V1.…”

“…Reducing the dimensionality of the problem eliminates many of the complications of the actual cortex, but retains the key feature that the properties of objects are preserved by the computations carried out by V1 (11). Imagine a 2D creature with a one-dimensional visual cortex.…”

The computational magic of the ventral stream

“…Separately, it has been argued that actual Gabor wavelets (with σ depending on ω) are a representation of the similarity group (in R 2 ). Stevens [30] develops an interesting and detailed argument for Gabor receptive fields in V1 to be implied by "invariance" to translations, scale and rotations. His Gabor wavelets have the form…”