Nonlinear and higher-order approaches to the encoding of natural scenes

Abstract

Linear operations can only partially exploit the statistical redundancies of natural scenes, and nonlinear operations are ubiquitous in visual cortex. However, neither the detailed function of the nonlinearities nor the higher-order image statistics are yet fully understood. We suggest that these complicated issues can not be tackled by one single approach, but require a range of methods, and the understanding of the crosslinks between the results. We consider three basic approaches: (i) State space descriptions can theoretically provide complete information about statistical properties and nonlinear operations, but their practical usage is confined to very low-dimensional settings. We discuss the use of representation-related state-space coordinates (multivariate wavelet statistics) and of basic nonlinear coordinate transformations of the state space (e.g., a polar transform). (ii) Indirect methods, like unsupervised learning in multi-layer networks, provide complete optimization results, but no direct information on the statistical properties, and no simple model structures. (iii) Approximation by lower-order terms of power-series expansions is a classical strategy that has not yet received broad attention. On the statistical side, this approximation amounts to cumulant functions and higher-order spectra (polyspectra), on the processing side to Volterra Wiener systems. In this context we suggest that an important concept for the understanding of natural scene statistics, of nonlinear neurons, and of biological pattern recognition can be found in AND-like combinations of frequency components. We investigate how the different approaches can be related to each other, how they can contribute to the understanding of cortical nonlinearities such as complex cells, cortical gain control, end-stopping and other extraclassical receptive field properties, and how we can obtain a nonlinear perspective on overcomplete representations and invariant coding in visual cortex.

Two-Dimensional Hermite Filters Simplify the Description of High-Order Statistics of Natural Images

“…Here we show that two-dimensional Hermite (TDH) filters, an orthogonal basis set with a high degree of symmetry, simplify the description of high-order statistics of natural images, both locally and over wide areas. The significance of this result is that high-order statistics carry the local features that distinguish natural images from Gaussian processes [3, 8, 17, 18, 29], but they are challenging to analyze because of their high dimensionality. By identifying a hidden symmetry in high-order statistics, TDH functions provide a kind of dimensional reduction, and therefore, a needed simplification.…”

“…Interestingly, [3] showed that this could be used to distinguish natural images from synthetic ones (including realistic computer-generated scenes), by applying linear classifiers to a feature space of wavelet coefficients. Other investigators have also used wavelet coefficients as a starting point, but focused on the extent to which wavelet coefficients are independent [17, 18]. Thus, the filter approach provides a useful characterization of natural image statistics -- but even with a filter-based approach, the number of parameters required to describe high-order image statistics is still large.…”

Sensorimotor representation and knowledge-based reasoning for spatial exploration and localisation

“…These features are of high perceptual relevance for object recognition (Attneave 1954, Mackworth and Morandi 1967, Biederman 1987. The extraction of these features can be provided by non-linear i2D-selective operators , which mimic the operations of nonlinear neurons in the visual cortex (Shapley 2004;Zetzsche and Nuding 2005). Figure 4 shows examples of these nonlinear sensory features.…”

Towards a state-space geometry of neural responses to natural scenes: A steady-state approach

“…This geometric distribution of responses can then be understood in accordance with the distribution of images that have been projected to different encoding spaces (such as those defined by visual filter outputs). This general approach has been used in theories of sparse coding [29] and the non-linear behavior of visual neurons [30][31]. By focusing on the full state-space geometry of the responses produced by an evoked potential (rather than simple features of the response), our experiments will show that it is possible to provide both a rational model of the signal as well as to provide an estimate of the information carried by that signal.…”