Neural model of visual stereomatching: slant, transparency and clouds

Abstract

Stereomatching of oblique and transparent surfaces is described using a model of cortical binocular 'tuned' neurons selective for disparities of individual visual features and neurons selective for the position, depth and 3D orientation of local surface patches. The model is based on a simple set of learning rules. In the model, monocular neurons project excitatory connection pathways to binocular neurons at appropriate disparities. Binocular neurons project excitatory connection pathways to appropriately tuned 'surface patch' neurons. The surface patch neurons project reciprocal excitatory connection pathways to the binocular neurons. Anisotropic intralayer inhibitory connection pathways project between neurons with overlapping receptive fields. The model's responses to simulated stereo image pairs depicting a variety of oblique surfaces and transparently overlaid surfaces are presented. For all the surfaces, the model (i) assigns disparity matches and surface patch representations based on global surface coherence and uniqueness, (ii) permits coactivation of neurons representing multiple disparities within the same image location, (iii) represents oblique slanted and tilted surfaces directly, rather than approximating them with a series of frontoparallel steps, (iv) assigns disparities to a cloud of points at random depths, like human observers and unlike Prazdny's (1985) method, and (v) causes globally consistent matches to override greedy local matches. The model represents transparency, unlike the model of Marr and Poggio (1976), and it assigns unique disparities, unlike the model of Prazdny.

A neural model for perceptual organization of 3D surfaces

“…We extend a neural model of visual stereomatching [13] that is conceptually similar to grouping models previously proposed for 2D stimuli [11], [12], [14]. The model contains three layers of neurons.…”

Models of receptive-field dynamics in visual cortex

“…The EXIN rules self-organize networks capable of representing multiple superimposed patterns, ambiguous patterns, overlapping patterns at different scales, and contextually constrained patterns starting from completely nonspecific afferent excitatory and lateral inhibitory pathway weights (Marshall, 1995). The EXIN afferent excitatory and lateral inhibitory synaptic plasticity rules together have been used to model the development of visual disparity selectivity (Marshall, 1990c), visual motion selectivity and grouping (Marshall, 1990a;Schmitt & Marshall, 1995, 1996, visual inertia (Hubbard & Marshall, 1994), visual motion integration in the aperture problem (Marshall, 1990a), visual length selectivity and end-stopping (Marshall, 1990b), visual depth perception from occlusion events (Marshall et al, 1996a), visual depth from motion parallax (Marshall, 1989), visual motion smearing (Martin & Marshall, 1993), visual orientation selectivity (Marshall, 1990d), and visual stereomatching (Marshall et al, 1996b).…”

Pre–attentive segmentation and correspondence in stereo

“…This correspondence problem is ill-posed as there are many possible false matches between individual dots, as illustrated in gure 1 for the wallpaper illusion. These observations led to a substantial research focus on solving the correspondence problem in early stereo vision, as exempli ed by the well-known model by Marr & Poggio (1976, 1979 in the 1970s and many other models since then (Prazdny 1985;Pollard et al 1985;Qian & Sejnowski 1989;Nasrabadi et al 1989;Geiger et al 1995;Marshall et al 1996;McLoughlin & Grossberg 1998;Watanabe & Fukushima 1999;Read 2002). These models employ cooperative algorithms (or indirectly, using optimizations or Bayesian approaches) that use contextual information to nd true binocular matches among all possible matches.…”