How Sensitive Is the Human Visual System to the Local Statistics of Natural Images?

Abstract

A key hypothesis in sensory system neuroscience is that sensory representations are adapted to the statistical regularities in sensory signals and thereby incorporate knowledge about the outside world. Supporting this hypothesis, several probabilistic models of local natural image regularities have been proposed that reproduce neural response properties. Although many such physiological links have been made, these models have not been linked directly to visual sensitivity. Previous psychophysical studies of sensitivity to natural image regularities focus on global perception of large images, but much less is known about sensitivity to local natural image regularities. We present a new paradigm for controlled psychophysical studies of local natural image regularities and compare how well such models capture perceptually relevant image content. To produce stimuli with precise statistics, we start with a set of patches cut from natural images and alter their content to generate a matched set whose joint statistics are equally likely under a probabilistic natural image model. The task is forced choice to discriminate natural patches from model patches. The results show that human observers can learn to discriminate the higher-order regularities in natural images from those of model samples after very few exposures and that no current model is perfect for patches as small as 5 by 5 pixels or larger. Discrimination performance was accurately predicted by model likelihood, an information theoretic measure of model efficacy, indicating that the visual system possesses a surprisingly detailed knowledge of natural image higher-order correlations, much more so than current image models. We also perform three cue identification experiments to interpret how model features correspond to perceptually relevant image features.

Human Sensitivity to Perturbations Constrained by a Model of the Natural Image Manifold

“…Our results seem to contradict reports which find that observers are very sensitive to deviations from naturalness (Gerhard et al, 2013): How would observers be equally sensitive to changes within the manifold of natural images and deviations away from that manifold? We find that sensitivity to perturbations within an approximation of the manifold of natural images can be well predicted by the part and object structure of the image that is captured by standard segmentation algorithms (see Figure 5), while deviations from naturalness might be more related to the fine structure of local statistics.…”

Contrast invariant tuning in human perception of image content

“…For example, Sebastian et al [46] superimposed target gratings on natural images resulting in stimuli with artificial foreground and natural background-still a very unnatural combination. Humans seem to be highly sensitive to deviations from natural appearance [47] and it is unclear to what extent deviations from natural appearance bias performance in psychophysical tasks involving natural images. Here, we used GANs to coordinate high-dimensional image manipulations and constrain them to remain approximately natural.…”

On combining visual perception and color structure based image quality assessment

“…These methods based on structure calculation are mostly implemented ignoring visual perception threshold (VPT) and color variation. Although the IQA model incorporating with VPT [14] [15] can reduce redundant information to evaluate the image quality, the variation of color structure is also extremely important for human perception [16] [17]. A significant advantage of color over the gray-level counterparts is that color contains a higher information level, which could provide a more accurately and visually appealing reproduction of the natural scene [18] [19].…”