Poggio and Poggio, 1984). They classified cells into a few discrete categories, although it now appears that Center for Neurobiology and Behavior Columbia University these categories represent idealized cases from a continuous distribution (LeVay and Voigt, 1988). More reNew York, New York 10032 cently, Ohzawa et al. (1990Ohzawa et al. ( , 1996 provided detailed quantitative mapping of binocular receptive fields of the We perceive the world in three-dimensions even though cat visual cortical cells and suggested models for simuthe input to our visual system, the images projected lating their responses. While these and many other exonto our two retinas, has only two spatial dimensions.periments have demonstrated the neural substrates for How is this accomplished? It is well known that the disparity coding at the earliest stage of binocular convisual system can infer the third dimension, depth, from vergence, they leave open the question of how a populaa variety of visual cues in the retinal images. One such tion of disparity selective cells could be used to compute cue is binocular disparity, the positional difference bedisparity maps from a pair of retinal images such as the tween the two retinal projections of a given point in stereograms used by Julez. What is needed, in addition space ( Figure 1). This positional difference results from to experimental investigations, is a computational thethe fact that the two eyes are laterally separated and ory (Marr, 1982; Churchland and Sejnowski, 1992) specitherefore see the world from two slightly different vanfying an algorithm for combining the neuronal signals tage points.into a meaningful computational scheme. The idea that retinal disparity contributes criticallyAlthough there have also been many computational to depth perception derives from the invention of the studies of stereo vision in the past, until recently, most stereoscope by Wheatstone in the 19th century, with studies had treated disparity computation mainly as a which he showed conclusively that the brain uses horimathematics or engineering problem while giving only zontal disparity to estimate the relative depths of objects secondary considerations to existing physiological data. in the world with respect to the fixation point, a process Part of this tradition stems from a belief advanced paraknown as stereoscopic depth perception or stereopsis.doxically by David Marr, one of the most original thinkers Because simple geometry provides relative depth given in vision research, that physiological details are not imretinal disparity, the problem of understanding stereo portant for understanding information processing tasks vision reduces to the question: How does the brain mea-(such as visual perception) at the systems level. Marr sure disparity from the two retinal images in the first (1982) argued that a real understanding will only come place?from an abstract computational analysis of how a particSince Wheatstone's discovery, students of vision sciular problem may be solved under certain mathematica...