Binocular 3D Object Recovery Using a Symmetry Prior

Abstract

We present a new algorithm for 3D shape reconstruction from stereo image pairs that uses mirror symmetry as a biologically inspired prior. 3D reconstruction requires some form of prior because it is an ill-posed inverse problem. Psychophysical research shows that mirror-symmetry is a key prior for 3D shape perception in humans, suggesting that a general purpose solution to this problem will have many applications. An approach is developed for finding objects that fit a given shape definition. The algorithm is developed for shapes with two orthogonal planes of symmetry, thus allowing for straightforward recovery of occluded portions of the objects. Two simulations were run to test: (1) the accuracy of 3D recovery, and (2) the ability of the algorithm to find the object in the presence of noise. We then tested the algorithm on the Children's Furniture Corpus, a corpus of stereo image pairs of mirror symmetric furniture objects. Runtimes and 3D reconstruction errors are reported and failure modes described.

Unifying Physics and Psychophysics on the Basis of Symmetry, Least-Action ≈ Simplicity Principle, and Conservation Laws ≈ Veridicality

“…Let the angle formed by points p1, C, and vp be ψ, and the angle formed by p2, C, and vp be ϕ. Then, the distances ||P1|| of P1 from C and ||P2|| of P2 from C are computed according to the following equations (see Michaux et al, 2017 for details of the derivation): ||P1|| = 2d⋅sin(ϕ)/sin(ϕ+ψ) ||P2|| = 2d⋅sin(ψ)/sin(ϕ+ψ)…”

“…Our most recent models work with real camera images. They also explicitly measure the difference between the 2D image produced by the 3D recovered shape and the 2D image that was used to perform the recovery (Michaux et al, 2016(Michaux et al, , 2017.…”

Human Symmetry Uncertainty Detected by a Self-Organizing Neural Network Map

“…Vertical mirror symmetry is a particularly salient form of visual symmetry [23][24][25], processed at early stages in human vision and producing greater or lesser detection reliability [23] depending on local features of the stimulus display with greater or lesser stimulus certainty. Shape symmetry is a visual property that attracts attention [18] and determines perceived volume [19][20][21][22] and perceptual salience [26] of objects represented in the two-dimensional image plane. Aesthetic judgment and choice preference [27,28] are influenced by symmetry, justifying biologically inspired models of symmetry perception in humans [29] under the light of the fact that symmetry is detected not only by primates but also by other species, such as insects, for example [30].…”

Failures of stereoscopic shape constancy over changes of viewing distance and size for bilaterally symmetric polyhedra

“…Algorithms for multiple-view geometry ( Hartley & Zisserman, 2003 ; Ma, Soatto, Kosecka, & Sastry, 2004 ) can be modified to recover 3D structure from a single image ( Hong et al, 2004 ; Ma et al, 2004 ). This is an active area of research in computer vision that is sometimes referred to as shape from symmetry or structure from symmetry (e.g., Franc¸ois et al, 2002 ; Gordon, 1989 ; Michaux, Kumar, Jayadevan, Delp, & Pizlo, 2017 ; Park et al, 2008 ; Sawada, Li., & Pizlo, 2011 ; Thrun & Wegbreit, 2005 ).…”