36Reconstructing three-dimensional (3D) scenes from two-dimensional (2D) retinal images is an ill-37 posed problem. Despite this, our 3D perception of the world based on 2D retinal images is 38 seemingly accurate and precise. The integration of distinct visual cues is essential for robust 3D 39 perception in humans, but it is unclear if this mechanism is conserved in non-human primates, 40 and how the underlying neural architecture constrains 3D perception. Here we assess 3D 41 perception in macaque monkeys using a surface orientation discrimination task. We find that 42 perception is generally accurate, but precision depends on the spatial pose of the surface and 43 available cues. The results indicate that robust perception is achieved by dynamically reweighting 44 the integration of stereoscopic and perspective cues according to their pose-dependent 45 reliabilities. They further suggest that 3D perception is influenced by a prior for the 3D orientation 46 statistics of natural scenes. We compare the data to simulations based on the responses of 3D 47 orientation selective neurons. The results are explained by a model in which two independent 48 neuronal populations representing stereoscopic and perspective cues (with perspective signals 49 from the two eyes combined using nonlinear canonical computations) are optimally integrated 50 through linear summation. Perception of combined-cue stimuli is optimal given this architecture. 51However, an alternative architecture in which stereoscopic cues and perspective cues detected 52 by each eye are represented by three independent populations yields two times greater precision 53 than observed. This implies that, due to canonical computations, cue integration for 3D perception 54 is optimized but not maximized. 55
56
Author summary 57Our eyes only sense two-dimensional projections of the world (like a movie on a screen), yet we 58 perceive the world in three dimensions. To create reliable 3D percepts, the human visual system 59 integrates distinct visual signals according to their reliabilities, which depend on conditions such 60 as how far away an object is located and how it is oriented. Here we find that non-human primates 61 similarly integrate different 3D visual signals, and that their perception is influenced by the 3D 62 orientation statistics of natural scenes. Cue integration is thus a conserved mechanism for 63 creating robust 3D percepts by the primate brain. Using simulations of neural population activity, 64 based on neuronal recordings from the same animals, we show that some computations which 65 occur widely in the brain facilitate 3D perception, while others hinder perception. This work 66addresses key questions about how neural systems solve the difficult problem of generating 3D 67 percepts, identifies a plausible neural architecture for implementing robust 3D vision, and reveals 68 how neural computation can simultaneously optimize and curb perception. 69 were linearly summed [10]). However, an alternative architecture in which stereoscopic as we...