Specular reflections and the estimation of shape from binocular disparity

Abstract

Binocular stereopsis is a powerful visual depth cue. To exploit it, the brain matches features from the two eyes' views and measures their interocular disparity. This works well for matte surfaces because disparities indicate true surface locations. However, specular (glossy) surfaces are problematic because highlights and reflections are displaced from the true surface in depth, leading to information that conflicts with other cues to 3D shape. Here, we address the question of how the visual system identifies the disparity information created by specular reflections. One possibility is that the brain uses monocular cues to identify that a surface is specular and modifies its interpretation of the disparities accordingly. However, by characterizing the behavior of specular disparities we show that the disparity signals themselves provide key information ("intrinsic markers") that enable potentially misleading disparities to be identified and rejected. We presented participants with binocular views of specular objects and asked them to report perceived depths by adjusting probe dots. For simple surfaces-which do not exhibit intrinsic indicators that the disparities are "wrong"-participants incorrectly treat disparities at face value, leading to erroneous judgments. When surfaces are more complex we find the visual system also errs where the signals are reliable, but rejects and interpolates across areas with large vertical disparities and horizontal disparity gradients. This suggests a general mechanism in which the visual system assesses the origin and utility of sensory signals based on intrinsic markers of their reliability.psychophysics | perception | gloss | texture | computational analysis S hiny objects such as sports cars, jewelry, and consumer electronics can be beautiful to look at. However, such objects pose a difficult challenge to the visual system: if all (or most) of the light reaching the eye comes from the reflections of other nearby objects, how does the viewer discern the object itself? This problem becomes more acute when viewing with two eyes. Unlike shading or texture markings, the positions of reflections relative to a specular (shiny) surface depend on the observer's viewpoint. This means that when the surface is viewed binocularly (i.e., from two viewpoints at the same time), corresponding reflections fall on different surface locations. In consequence, the binocular disparities created by specular reflections indicate depth positions displaced from the object's physical surface (1, 2) and the 3D shape specified by disparity can be radically different from the true shape of the object. For special cases, such as an ideal planar mirror, the visual system could not, even in principle, estimate the true depths of the surface from the reflections. However, for more complex shapes, such as a polished metal kettle, we rarely encounter problems judging shape. Most models of biological vision place heavy weight on binocular disparity cues, whereas artificial systems often rely almost exclusively ...

Investigating perceptual qualities of static surface appearance using real materials and displayed images

“…and shape are influential in distinguishing materials by providing relevant visual information. Furthermore, as has been shown previously (Doerschner et al, 2011;Muryy, Welchman, Blake, & Fleming, 2013;Wendt, Faul, & Mausfeld, 2008), judgments of specular reflectance are affected by both binocular disparity and motion information. Many studies have analyzed this by presenting the two stimuli simultaneously.…”

Key characteristics of specular stereo

“…textured object, we can think of this process as two eyes viewing a particular texture element of the surface of the object, with the brain charged with establishing correspondence between the retinal projections of the texture elements in the two images. In the specular case, we assume (like previous work on specular stereo; Muryy et al, 2013) that the definition of correspondence is essentially equivalent; namely, that the visual system seeks the image feature in one eye that matches the same image feature in the other eye. Thus, the basic task is the same (seeking correspondence between image features); however, in the specular case, these features originate from the reflection of the environment illumination map, rather than markings on surface itself.…”

“…Finally, we show that the distribution of ortho-epipolar disparities is related to surface geometry, providing a constraint when estimating the curvature of the viewed object. Thereby we show that even though specular stereo signals do not support direct perceptual estimates of the physical shape of an object (Muryy, Welchman, Blake, & Fleming, 2013), specular disparity fields do carry information about the intimate relations between the viewing geometry and surface topography which could potentially be exploited by humans and artificial systems.…”

“…Here, we seek to characterize the conditions that lead to the displacement of specular reflections. We do this as a means of understanding why human observers treat reflections as though they are true surface markings when judging depths (Muryy, Welchman, Blake, & Fleming, 2013). In the process, we also quantify other respects in which specular reflections deviate from standard stereopsis.…”

Perception of highlight disparity at a distance in consumer head-mounted displays

“…Furthermore, sharing radiance also causes specular highlights to appear at the same convergence angle as the reflecting object's surface. This is in contrast with physical reality, where specular reflections appear at a depth different than that of the reflecting surface [Kirschmann 1895;Muryy et al 2013], an effect called highlight disparity. Again, this may cause observers to underestimate surface glossiness [Wendt et al 2010] and even reduce authenticity [Wendt et al 2008].…”