Perception viewed as an inverse problem

Abstract

The modern study of perception began when Fechner published his 'Elements of Psychophysics' in 1860. This book has guided most perception research ever since. It has become increasingly clear that there are problems with Fechner's approach, which assumes that the percept is completely determined by the sensory input. Fechner's approach cannot explain the processes that allow our percepts to be veridical. Post-Fechnerian schools (Helmholtzian, Structural, Gestalt and Gibsonian) have tried to deal with this problem, but have not been successful. An alternative to the Fechnerian approach is required. This paper describes an alternative that has been developing over the last 20 years within the computer vision community. It treats perceptual interpretation as a solution of an inverse problem that depends critically on the operation of a priori constraints. Contemporary research, which adopted this approach, has concentrated on verifying the usefulness of Bayesian and standard regularization methods. This paper takes the next step; it discusses theoretical and empirical aspects of studying human perception as an inverse problem. It reviews the literature that illustrates the power of the inverse problem approach. This review leads to the suggestion that progress in the study of perception will benefit if the inverse approach were to be adopted by experimentalists, as well as by the computational modelers, who have been actively exploring its potential to date.

How biological vision succeeds in the physical world

“…Indeed, generating veridical perceptions seems impossible given the uncertain significance of information conveyed by retinal stimuli (Fig. 1), even when the constraints of physics that define the world are taken into account (10)(11)(12).…”

“…The problematic relationship between visual stimuli and the physical world was recognized by Ptolemy in the 2nd century, Alhazen in the 11th century, Berkeley in the 18th century, Helmholtz in the 19th century, and many others since (2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12). To explain how accurate perceptions and behaviors could arise from stimuli that cannot specify their sources, Helmholtz, arguably the most influential figure over this history, proposed that observers augmented the information in retinal stimuli by making "unconscious inferences" about the world based on past experience.…”

Binocular shape constancy from novel views: The role of a priori constraints

“…There are two general methods of incorporating a priori constraints into models of perceptual reconstruction: probabilistic, involving the likelihood (Bayes's) principle, and deterministic, involving the simplicity principle (e.g., Kersten, Mamassian, & Yuille, 2004;Pizlo, 2001;Pomerantz & Kubovy, 1986). Each of these methods can be reformulated in the language of the other, and therefore, the choice between these two methods depends largely on the assumptions about nature of the constraints.…”

“…As a result, the inverse problem of reconstructing a 3-D shape from its 2-D image has infinitely many solutions. When two or more images are available, the 3-D scene may, in principle, be uniquely reconstructed, but the reconstruction tends to be unstable in the presence of visual noise (Pizlo, 2001;Poggio, Torre, & Koch, 1985). The most effective way to overcome the computational difficulty of shape reconstruction is to impose a priori constraints on the family of possible 3-D interpretations.…”

Any Pair of 2D Curves Is Consistent with a 3D Symmetric Interpretation

“…However, since 3D symmetric interpretations exist for every 2D image, including images of an asymmetric 3D shape, one can never be sure whether the recovered 3D symmetry is the correct 3D interpretation. Clearly, 3D recovery in human and computer vision must be treated as an ill-posed inverse problem, whose solution involves tools of a regularization theory or Bayesian inference [1,29,30]. When inverse problems are solved by the human visual system, the retinal information is combined with a priori constraints, and symmetry is only one of several constraints that are being used by the visual system.…”