Slant from texture and disparity cues: Optimal cue combination

Hillis, James; Watt, Simon J.; Landy, Michael S.; Banks, Martin S.

doi:10.1167/4.12.1

Cited by 394 publications

(493 citation statements)

References 46 publications

Supporting

Mentioning

454

Contrasting

Unclassified

Order By: Relevance

“…More broadly, this can be understood in terms of optimal cue combination approaches to perception (e.g. Hillis, Watt, Landy, & Banks, 2004;Jacobs, 2002;Knill & Saunders, 2003), and is also consistent with other proposals that recognition may involve both image-based and structural description representations (e.g., Foster & Gilson, 2002;Hummel, 2013;Hummel & Stankiewicz, 1996).…”

Section: Discussionsupporting

confidence: 85%

Stereo Disparity Facilitates View Generalization during Shape Recognition for Solid Multipart Objects

Cristino

Davitt

Hayward

et al. 2015

Quarterly Journal of Experimental Psychology

View full text Add to dashboard Cite

Current theories of object recognition in human vision make different predictions about whether the recognition of complex, multipart objects should be influenced by shape information about surface depth orientation and curvature derived from stereo disparity. We examined this issue in five experiments using a recognition memory paradigm in which observers (N = 134) memorized and then discriminated sets of 3D novel objects at trained and untrained viewpoints under either mono or stereo viewing conditions. In order to explore the conditions under which stereo-defined shape information contributes to object recognition we systematically varied the difficulty of view generalization by increasing the angular disparity between trained and untrained views. In one series of experiments, objects were presented from either previously trained views or untrained views rotated (15°, 30°, or 60°) along the same plane. In separate experiments we examined whether view generalization effects interacted with the vertical or horizontal plane of object rotation across 40° viewpoint changes. The results showed robust viewpoint-dependent performance costs: Observers were more efficient in recognizing learned objects from trained than from untrained views, and recognition was worse for extrapolated than for interpolated untrained views. We also found that performance was enhanced by stereo viewing but only at larger angular disparities between trained and untrained views. These findings show that object recognition is not based solely on 2D image information but that it can be facilitated by shape information derived from stereo disparity.

show abstract

Section: Discussionsupporting

confidence: 85%

Stereo Disparity Facilitates View Generalization during Shape Recognition for Solid Multipart Objects

Cristino

Davitt

Hayward

et al. 2015

Quarterly Journal of Experimental Psychology

View full text Add to dashboard Cite

show abstract

“…Several works have reported that humans perform near-optimal cue integration in a variety of settings (1)(2)(3)(4)(5)(6)(7)(8). It is, therefore, essential that the combination of inputs that leads to the multiplicative rule in an attractor network also results in optimal cue integration.…”

Section: Discussionmentioning

confidence: 99%

“…In some cases, humans can perform these inferences optimally, as in multi-cue or multisensory integration (1)(2)(3)(4)(5)(6)(7)(8). For complex tasks, such as object recognition, action perception, and object tracking, the computations required for optimal inference are intractable, which implies that humans must use approximate inferences (9)(10)(11).…”

mentioning

confidence: 99%

Bayesian sampling in visual perception

Moreno-Bote

Knill

Pouget³

2011

Proc. Natl. Acad. Sci. U.S.A.

149

135

View full text Add to dashboard Cite

It is well-established that some aspects of perception and action can be understood as probabilistic inferences over underlying probability distributions. In some situations, it would be advantageous for the nervous system to sample interpretations from a probability distribution rather than commit to a particular interpretation. In this study, we asked whether visual percepts correspond to samples from the probability distribution over image interpretations, a form of sampling that we refer to as Bayesian sampling. To test this idea, we manipulated pairs of sensory cues in a bistable display consisting of two superimposed moving drifting gratings, and we asked subjects to report their perceived changes in depth ordering. We report that the fractions of dominance of each percept follow the multiplicative rule predicted by Bayesian sampling. Furthermore, we show that attractor neural networks can sample probability distributions if input currents add linearly and encode probability distributions with probabilistic population codes.Bayesian inference | neuronal network | neuronal noise | perceptual bistability T here is mounting evidence that neural circuits can implement probabilistic inferences over sensory, cognitive, or motor variables. In some cases, humans can perform these inferences optimally, as in multi-cue or multisensory integration (1-8). For complex tasks, such as object recognition, action perception, and object tracking, the computations required for optimal inference are intractable, which implies that humans must use approximate inferences (9-11). One approximate scheme that is particularly appealing from a biological point of view is sampling. Consider as an example the problem of object recognition. The goal of the inference in this case would be to compute the probability over object identities given the image. Although this probability distribution may be difficult to compute explicitly, one can often design algorithms to generate samples from the distribution, allowing one to perform approximate inference (12, 13). Some human cognitive choice behaviors suggest that the nervous system implements sampling. However, whether the same is true for lowlevel perceptual processing is currently unknown.Stimuli that lead to bistable percepts (14-18), like the Necker cube, provide a tractable experimental preparation for testing the sampling hypothesis. With such stimuli, perception alternates stochastically between two possible interpretations, a behavior consistent with sampling as suggested by several works (16,19,20). However, the key question is what probability distribution is being sampled. If the brain uses sampling for Bayesian inference, neural circuits should sample from an internal probability distribution on possible stimulus interpretations that are conditioned on the available sensory data, the so-called posterior distribution. This distribution places important constraints on the distributions of perceptual states for bistable stimuli.To test this idea, we used stimuli composed of two dr...

show abstract

“…It is well known that the visual system can use these texture compression cues to estimate 3D shape (1)(2)(3)(4)(5)(6)(7)(8)(9). What is not known, however, is how the visual system measures the compression at each point in the image.…”

mentioning

confidence: 99%

Estimation of 3D shape from image orientations

Fleming

Holtmann-Rice

Bülthoff

2011

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

One of the main functions of vision is to estimate the 3D shape of objects in our environment. Many different visual cues, such as stereopsis, motion parallax, and shading, are thought to be involved. One important cue that remains poorly understood comes from surface texture markings. When a textured surface is slanted in 3D relative to the observer, the surface patterns appear compressed in the retinal image, providing potentially important information about 3D shape. What is not known, however, is how the brain actually measures this information from the retinal image. Here, we explain how the key information could be extracted by populations of cells tuned to different orientations and spatial frequencies, like those found in the primary visual cortex. To test this theory, we created stimuli that selectively stimulate such cell populations, by "smearing" (filtering) images of 2D random noise into specific oriented patterns. We find that the resulting patterns appear vividly 3D, and that increasing the strength of the orientation signals progressively increases the sense of 3D shape, even though the filtering we apply is physically inconsistent with what would occur with a real object. This finding suggests we have isolated key mechanisms used by the brain to estimate shape from texture. Crucially, we also find that adapting the visual system's orientation detectors to orthogonal patterns causes unoriented random noise to look like a specific 3D shape. Together these findings demonstrate a crucial role of orientation detectors in the perception of 3D shape.shape perception | surface perception | orientation field | complex cells W hen we look at a textured object, the projection of the surface markings into the retinal image compresses them in ways that can indicate the object's 3D shape. This compression has two distinct causes. The first cause is distance-dependent: when a surface patch is moved further away from the eye, the texture shrinks isotropically in the image as a function of the distance. The second cause of compression is foreshortening: when a surface is slanted relative to the line of sight, the texture is anisotropically compressed along the direction of the slant, with greater slant leading to greater compression.It is well known that the visual system can use these texture compression cues to estimate 3D shape (1-9). What is not known, however, is how the visual system measures the compression at each point in the image. A crucial stage in any theory of 3D vision must include an explanation of how the visual system extracts the key information from the image. At present, there is an explanatory gap between the known response properties of cells early in the visual processing hierarchy, which measure local 2D image features (10-16), and cells higher in the processing stream, which respond to various 3D shape properties (17-24). How does the brain put the measurements made in the primary visual cortex (V1) to good use to arrive at an estimate of 3D shape?Estimating the extent and direction of ...

show abstract

Slant from texture and disparity cues: Optimal cue combination

Cited by 394 publications

References 46 publications

Stereo Disparity Facilitates View Generalization during Shape Recognition for Solid Multipart Objects

Stereo Disparity Facilitates View Generalization during Shape Recognition for Solid Multipart Objects

Bayesian sampling in visual perception

Estimation of 3D shape from image orientations

Contact Info

Product

Resources

About