Part-Based Representations of Visual Shape and Implications for Visual Cognition

Abstract

Human vision organizes object shapes in terms of parts and their spatial relationships.Converging experimental evidence suggests that parts are computed rapidly and early in visual processing. We review theories of how human vision parses shapes. In particular, we discuss the minima rule for finding part boundaries on shapes, geometric factors for creating part cuts, and a theory of part salience. We review empirical evidence that human vision parses shapes into parts, and show that parts-based representations explain various aspects of our visual cognition, including figure-ground assignment, judgments of shape similarity, memory for shapes, visual search for shapes, the perception of transparency, and the allocation of visual attention to objects. From Fragments to Objects: Segmentation and Grouping in Vision2

Visual Representation of Contour and Shape

“…1 Similarly, De Winter & Wagemans (2008b) found that when participants are asked simply to mark "salient points" along the bounding contours of 2D shapes-without being required to replicate the shape-they are again most likely to pick local maxima of curvature. As we will see, curvature extrema play an important role in modern theories of shape representation as well (Hoffman & Richards, 1984;Richards, Dawson & Whittington, 1986;Leyton, 1989;Hoffman & Singh, 1997;Singh & Hoffman, 2001;De Winter & Wagemans, 2006;2008a;Cohen & Singh, 2007). But why should curvature maxima be the most informative points along a contour?…”

“…Specifically, although the minima rule provides a number of candidate part boundaries (namely, the negative minima of curvature), it does not indicate how these boundaries should be paired to form part cuts that segment the shape. Furthermore, even in shapes containing exactly two negative minima, simply connecting these two minima does not necessarily yield intuitive part segmentations (see e.g., Singh, Seyranian & Hoffman, 1999;Singh & Hoffman, 2001 for examples). The basic limitation of the minima rule stems from the fact that localizing negative minima of curvature involves only the local geometry of the bounding contour of the shape, but not the nonlocal geometry its interior region (see Section 4 for more on this important distinction).…”

“…Because of the contributions of such nonlocal regionbased factors, it is possible to have negative minima on a shape that do not correspond to perceptual part boundaries ( Figure 5a) and, conversely, to have perceptual part boundaries that do not correspond to negative minima (Figure 5b). In order to address such limitations, researchers have proposed a number of additional geometric factors for segmenting objects into parts: limbs and necks (Siddiqi, Tresness & Kimia 1996), convexity (Latecki & Lakamper, 1999;Rosin, 2000), a preference for shorter cuts , local symmetry, good continuation (Singh & Hoffman, 2001), as well as cognitive factors based on object knowledge (De Winter & Wagemans, 2006). And each of these factors has indeed been shown to play a role in part segmentation.…”

An Integrated Bayesian Approach to Shape Representation and Perceptual Organization

“…[30]), there is substantial evidence that human object recognition uses structural representations based on combinations of shape parts [3,1]. But though many factors are known to influence the decomposition of shapes into parts [9,41,45,46], we still lack a comprehensive account of part decomposition. A simple and principled account of part decomposition is directly entailed by the Bayesian approach to shape representation, encompassing several well-known part-decomposition rules as sideeffects.…”

“…the fact that part boundaries tend to occur near minima, and also its failures, e.g. part boundaries that occur where there are no curvature minima and curvature minima that are not perceived as part boundaries (see [45]). Similarly, regions of common axial ownership tend to be relatively convex, subsuming the part convexity principle [38], and part cuts tend to be relatively short, subsuming the short-cut rule [46].…”