Perception of depth is based on a variety of cues, with binocular disparity and motion parallax generally providing more precise depth information than pictorial cues. Much is known about how neurons in visual cortex represent depth from binocular disparity or motion parallax, but little is known about the joint neural representation of these depth cues. We recently described neurons in the middle temporal (MT) area that signal depth sign (near vs far) from motion parallax; here, we examine whether and how these neurons also signal depth from binocular disparity. We find that most MT neurons in rhesus monkeys (Macaca Mulatta) are selective for depth sign based on both disparity and motion parallax cues. However, the depth-sign preferences (near or far) are not always aligned: 56% of MT neurons have matched depth-sign preferences ("congruent" cells) whereas the remaining 44% of neurons prefer near depth from motion parallax and far depth from disparity, or vice versa ("opposite" cells). For congruent cells, depth-sign selectivity increases when disparity cues are added to motion parallax, but this enhancement does not occur for opposite cells. This suggests that congruent cells might contribute to perceptual integration of depth cues. We also found that neurons are clustered in MT according to their depth tuning based on motion parallax, similar to the known clustering of MT neurons for binocular disparity. Together, these findings suggest that area MT is involved in constructing a representation of 3D scene structure that takes advantage of multiple depth cues available to mobile observers.

Observer translation produces differential image motion between objects that are located at different distances from the observer's point of fixation [motion parallax (MP)]. However, MP can be ambiguous with respect to depth sign (near vs far), and this ambiguity can be resolved by combining retinal image motion with signals regarding eye movement relative to the scene. We have previously demonstrated that both extra-retinal and visual signals related to smooth eye movements can modulate the responses of neurons in area MT of macaque monkeys, and that these modulations generate neural selectivity for depth sign. However, the neural mechanisms that govern this selectivity have remained unclear. In this study, we analyze responses of MT neurons as a function of both retinal velocity and direction of eye movement, and we show that smooth eye movements modulate MT responses in a systematic, temporally precise, and directionally specific manner to generate depth-sign selectivity. We demonstrate that depth-sign selectivity is primarily generated by multiplicative modulations of the response gain of MT neurons. Through simulations, we further demonstrate that depth can be estimated reasonably well by a linear decoding of a population of MT neurons with response gains that depend on eye velocity. Together, our findings provide the first mechanistic description of how visual cortical neurons signal depth from MP.

As an observer translates, objects lying at different distances from the observer have differential image motion on the retina (motion parallax). It is well established psychophysically that humans perceive depth rather precisely from motion parallax and that extraretinal signals may be used to correctly perceive the sign of depth (near vs far) when binocular and pictorial depth cues are absent or weak. However, the neural basis for this capacity remains poorly understood. We have shown previously that neurons in the macaque middle temporal (MT) area combine retinal image motion with smooth eye movement command signals to signal depth sign from motion parallax. However, those studies were performed in animals that were required simply to track a visual target, thus precluding direct comparisons between neural activity and behavior. Here, we examine the activity of MT neurons in rhesus monkeys that were trained to discriminate depth sign based on motion parallax, in the absence of binocular disparity and pictorial depth cues. We find that the most sensitive MT neurons approach behavioral sensitivity, whereas the average neuron is twofold to threefold less sensitive than the animal. We also find that MT responses are predictive of perceptual decisions (independent of the visual stimulus), consistent with a role for MT in providing sensory signals for this behavior. Our findings suggest that, in addition to its established roles in processing stereoscopic depth, area MT is well suited to contribute to perception of depth based on motion parallax.