The “independent components” of natural scenes are edge filters

Abstract

It has previously been suggested that neurons with line and edge selectivities found in primary visual cortex of cats and monkeys form a sparse, distributed representation of natural scenes, and it has been reasoned that such responses should emerge from an unsupervised learning algorithm that attempts to find a factorial code of independent visual features. We show here that a new unsupervised learning algorithm based on information maximization, a nonlinear "infomax" network, when applied to an ensemble of natural scenes produces sets of visual filters that are localized and oriented. Some of these filters are Gabor-like and resemble those produced by the sparseness-maximization network. In addition, the outputs of these filters are as independent as possible, since this infomax network performs Independent Components Analysis or ICA, for sparse (super-gaussian) component distributions. We compare the resulting ICA filters and their associated basis functions, with other decorrelating filters produced by Principal Components Analysis (PCA) and zero-phase whitening filters (ZCA). The ICA filters have more sparsely distributed (kurtotic) outputs on natural scenes. They also resemble the receptive fields of simple cells in visual cortex, which suggests that these neurons form a natural, information-theoretic coordinate system for natural images.

Robust Face Recognition Based on Part-Based Localized Basis Images

“…PCA and ICA architecture II basis images, as shown in Figure 1 (a) and (b), respectively, display global properties in the sense that they assign significant weights to potentially all the pixels. This accords with the fact that PCA basis images are just scaled versions of global Fourier filters [12]. In contrast, ICA architecture I basis images are spatially more localized.…”

“…The FastICA method computes independent components by maximizing nonGaussianity of whitened data distribution using a kurtosis maximization process. The kurtosis measures the non-Gaussianity and the sparseness of the face representations [12]. The FastICA algorithm is briefly described as follows.…”

“…The ICA architecture II basis images display global properties as shown in Figure 1 (c). Since the kurtosis maximization yields sparseness of basis images, the ICA architecture I basis images are spatially localized edge filters [12]. However, they do not display perfectly local characteristics in the sense that pixels that do not belong to locally salient feature regions still have some nonzero weight values.…”

“…These regions correspond to important facial feature regions such as eyes, eye brow, nose and lips. Note that ICA architectureⅠproduces basis images that are localized edge filters [12], and they correspond to meaningful facial feature regions.…”

Sparse Coding with Invariance Constraints

“…Following this concept, wavelet-like features resembling the receptive £elds of V1 cells have been derived either by imposing sparse overcomplete representations [9] or statistical independence as in independent component analysis [2]. Extensions for complex cells [7,6] and spatiotemporal receptive £elds were shown [5].…”