Binocular Mechanisms of 3D Motion Processing

Cormack, Lawrence K.; Czuba, Thaddeus B.; Knöll, Jonas; Huk, Alexander C.

doi:10.1146/annurev-vision-102016-061259

Cited by 38 publications

(44 citation statements)

References 109 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We found optimal integration of stereoscopic and perspective cues, consistent with previous human results [4, 5]. However, previous studies did not consider that combined-cue stimuli actually contain three cues: stereoscopic, left eye perspective, and right eye perspective [34]. A distinction between left and right eye perspective cues may seem surprising, but the two retinal projections of 3D stimuli can differ enough to yield significantly different discrimination performance [35].…”

Section: Resultssupporting

confidence: 88%

Optimized but not maximized cue integration for 3D visual perception

Chang

Kim

Thompson

et al. 2019

Preprint

View full text Add to dashboard Cite

36Reconstructing three-dimensional (3D) scenes from two-dimensional (2D) retinal images is an ill-37 posed problem. Despite this, our 3D perception of the world based on 2D retinal images is 38 seemingly accurate and precise. The integration of distinct visual cues is essential for robust 3D 39 perception in humans, but it is unclear if this mechanism is conserved in non-human primates, 40 and how the underlying neural architecture constrains 3D perception. Here we assess 3D 41 perception in macaque monkeys using a surface orientation discrimination task. We find that 42 perception is generally accurate, but precision depends on the spatial pose of the surface and 43 available cues. The results indicate that robust perception is achieved by dynamically reweighting 44 the integration of stereoscopic and perspective cues according to their pose-dependent 45 reliabilities. They further suggest that 3D perception is influenced by a prior for the 3D orientation 46 statistics of natural scenes. We compare the data to simulations based on the responses of 3D 47 orientation selective neurons. The results are explained by a model in which two independent 48 neuronal populations representing stereoscopic and perspective cues (with perspective signals 49 from the two eyes combined using nonlinear canonical computations) are optimally integrated 50 through linear summation. Perception of combined-cue stimuli is optimal given this architecture. 51However, an alternative architecture in which stereoscopic cues and perspective cues detected 52 by each eye are represented by three independent populations yields two times greater precision 53 than observed. This implies that, due to canonical computations, cue integration for 3D perception 54 is optimized but not maximized. 55 56 Author summary 57Our eyes only sense two-dimensional projections of the world (like a movie on a screen), yet we 58 perceive the world in three dimensions. To create reliable 3D percepts, the human visual system 59 integrates distinct visual signals according to their reliabilities, which depend on conditions such 60 as how far away an object is located and how it is oriented. Here we find that non-human primates 61 similarly integrate different 3D visual signals, and that their perception is influenced by the 3D 62 orientation statistics of natural scenes. Cue integration is thus a conserved mechanism for 63 creating robust 3D percepts by the primate brain. Using simulations of neural population activity, 64 based on neuronal recordings from the same animals, we show that some computations which 65 occur widely in the brain facilitate 3D perception, while others hinder perception. This work 66addresses key questions about how neural systems solve the difficult problem of generating 3D 67 percepts, identifies a plausible neural architecture for implementing robust 3D vision, and reveals 68 how neural computation can simultaneously optimize and curb perception. 69 were linearly summed [10]). However, an alternative architecture in which stereoscopic as we...

show abstract

Section: Resultssupporting

confidence: 88%

Optimized but not maximized cue integration for 3D visual perception

Chang

Kim

Thompson

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…In other words, if the motion system does not take into account viewing distance between the moving object and the observer then Bayesian estimation would lead to implausible 3D motion predictions. A model with a 3D motion prior on the other hand makes ecologically plausible 3D velocity predictions [33,34,39].…”

Section: Spherical Motion Priormentioning

confidence: 99%

On the Aperture Problem of Binocular 3D Motion Perception

Lages

Heron

2019

Vision

View full text Add to dashboard Cite

Like many predators, humans have forward-facing eyes that are set a short distance apart so that an extensive region of the visual field is seen from two different points of view. The human visual system can establish a three-dimensional (3D) percept from the projection of images into the left and right eye. How the visual system integrates local motion and binocular depth in order to accomplish 3D motion perception is still under investigation. Here, we propose a geometric-statistical model that combines noisy velocity constraints with a spherical motion prior to solve the aperture problem in 3D. In two psychophysical experiments, it is shown that instantiations of this model can explain how human observers disambiguate 3D line motion direction behind a circular aperture. We discuss the implications of our results for the processing of motion and dynamic depth in the visual system.

show abstract

“…However, in this paper we consider only a moving observer (or eye) A coordinate framework for optic flow and disparity Glennerster and Read in a static 3D world so that binocular vision and optic flow are equivalent in this regard. Note that we are not considering the integration of disparity and motion information as a binocular observer moves (Cormack et al, 2017). We are considering a unified framework for describing flow across the whole sphere, both near the epipole (traditionally, the part of the sphere that is considered by optic flow researchers) and around 90 degrees from the epipole (traditionally, the part of the sphere that is considered by binocular vision researchers).…”

Section: Introductionmentioning

confidence: 99%

2-D coordinate frames for optic flow and disparity

Glennerster

Read

2016

Journal of Vision

View full text Add to dashboard Cite

Optic flow is two dimensional, but no special qualities are attached to one or other of these dimensions. For binocular disparity, on the other hand, the terms 'horizontal ' and 'vertical' disparities are commonly used. This is odd, since binocular disparity and optic flow describe essentially the same thing. The difference is that, generally, people tend to fixate relatively close to the direction of heading as they move, meaning that fixation is close to the optic flow epipole, whereas, for binocular vision, fixation is close to the head-centric midline, i.e. approximately 90 degrees from the binocular epipole. For fixating animals, some separations of flow may lead to simple algorithms for the judgement of surface structure and the control of action. We consider the following canonical flow patterns that sum to produce overall flow: (i) 'towards' flow, the component of translational flow produced by approaching (or retreating from) the fixated object, which produces pure radial flow on the retina; (ii) 'sideways' flow, the remaining component of translational flow, which is produced by translation of the optic centre orthogonal to the cyclopean line of sight and (iii) 'vergence' flow, rotational flow produced by a counter-rotation of the eye in order to maintain fixation.A general flow pattern could also include (iv) 'cyclovergence' flow, produced by rotation of one eye relative to the other about the line of sight. We consider some practical advantages of dividing up flow in this way when an observer fixates as they move. As in some previous treatments, we suggest that there are certain tasks for which it is sensible to consider 'towards' flow as one component and 'sideways' + 'vergence' flow as another.A coordinate framework for optic flow and disparity Glennerster and Read Author Summary "Optic flow" refers to changes in the visual images we receive as we move through a scene.For example, as we drive along a street, the buildings flow past us and distant objects expand in our field of view. This information can tell us both about how we are moving and about the 3D structure of the scene. Conversely, "binocular disparity" refers to the differences between the views seen by our two eyes due to their slightly different positions in their head.Binocular disparity enables us to perceive the distance to objects, even while stationary. Both flow and disparity are rich sources of information, both for humans and other animals, and for robot systems such as autonomous drones and cars. Generally, they have been studied separately, and a separate set of vocabulary has been developed for each. Yet mathematically the two are fundamentally related: optic flow compares views seen by a single, moving eye at two different points in time, while binocular disparity compares views seen simultaneously by two eyes at different positions. Here, we develop a common language for describing both.It is particularly appropriate for eyes that fixate as they move, a universal feature of biological visual systems.A coordinate framewor...

show abstract

Binocular Mechanisms of 3D Motion Processing

Cited by 38 publications

References 109 publications

Optimized but not maximized cue integration for 3D visual perception

Optimized but not maximized cue integration for 3D visual perception

On the Aperture Problem of Binocular 3D Motion Perception

2-D coordinate frames for optic flow and disparity

Contact Info

Product

Resources

About