Visual shape perception as Bayesian inference of 3D object-centered shape representations.

Abstract

Despite decades of research, little is known about how people visually perceive object shape. We hypothesize that a promising approach to shape perception is provided by a "visual perception as Bayesian inference" framework which augments an emphasis on visual representation with an emphasis on the idea that shape perception is a form of statistical inference. Our hypothesis claims that shape perception of unfamiliar objects can be characterized as statistical inference of 3D shape in an object-centered coordinate system. We describe a computational model based on our theoretical framework, and provide evidence for the model along two lines. First, we show that, counterintuitively, the model accounts for viewpoint-dependency of object recognition, traditionally regarded as evidence against people's use of 3D object-centered shape representations. Second, we report the results of an experiment using a shape similarity task, and present an extensive evaluation of existing models' abilities to account for the experimental data. We find that our shape inference model captures subjects' behaviors better than competing models. Taken as a whole, our experimental and computational results illustrate the promise of our approach and suggest that people's shape representations of unfamiliar objects are probabilistic, 3D, and object-centered. (PsycINFO Database Record

Shape from Contour: Computation and Representation

“…For decades, these experiments have been taken as support for the view-based theory of object recognition. However, Erdogan & Jacobs (2017) have recently demonstrated that a 3D modelbased theory of object representation would also be expected to generate viewpoint-dependent recognition performance, once sensing noise and observer uncertainty about object structure are taken into account.…”

“…There is also some evidence that 3D structural models may better account for behavioral data than discriminative models, including recent DNN models, even if the latter currently serve as our best predictors of neural response in higher object areas of monkey and human. Erdogan & Jacobs (2017) recently found that a generative, Bayesian structural 3D object model provides better predictions of human judgments of 3D object shape similarity than discriminative models, including DNNs trained on ImageNet.…”

Efficient inverse graphics in biological face processing

“…To explain these inferences, early vision scientists proposed that scene analysis proceeds by inverting causal generative models, also known as "analysis-by-synthesis" or "inverse graphics." Approaches to inverse graphics have been considered for decades in computational vision (3,(5)(6)(7)(8), and these models have some behavioral support (9). However, inference in these models has traditionally been based on top-down stochastic search algorithms, such as Markov chain Monte Carlo (MCMC), which are highly iterative and implausibly slow.…”

Skeletal descriptions of shape provide unique perceptual information for object recognition

“…Skeletal similarity between every object was calculated in 3D, object-centered, space as the mean Euclidean distance between each point on one skeleton and the closest point on the second skeleton following maximal alignment (see Methods). We chose to test a 3D skeletal description because of behavioral 48 and neural 49 evidence for 3D object-centered representations in the visual system, which include a sensitivity to 3D skeletal structures 36,37 .…”