Contours of Vision: Towards a Compositional Semantics of Perception

Abstract

Mental capacities for perceiving, remembering, thinking, and planning involve the processing of structured mental representations. A compositional semantics of such representations would explain how the content of any given representation is determined by the contents of its constituents and their mode of combination. While many have argued that semantic theories of mental representations would have broad value for understanding the mind, there have been few attempts to develop such theories in a systematic and empirically constrained way. This paper contributes to that end by developing a semantics for a 'fragment' of our mental representational system: the visual system's representations of the bounding contours of objects. At least three distinct kinds of composition are involved in such representations: 'concatenation', 'feature composition', and 'contour composition'. I sketch the constraints on and semantics of each of these. This account has three principal payoffs. First, it models a working framework for compositionally ascribing structure and content to perceptual representations, while highlighting core kinds of evidence that bear on such ascriptions. Second, it shows how a compositional semantics of perception can be compatible with holistic, or Gestalt, phenomena, which are often taken to show that the whole percept is 'other than the sum of its parts'. Finally, the account illuminates the format of a key type of perceptual representation, bringing out the ways in which contour representations exhibit domain-specific form of the sort that is typical of structured icons such as diagrams and maps, in contrast to typical discursive representations of logic and language.

A Pluralist Perspective on Shape Constancy

“…14 Note, finally, that skeletal representation presupposes other capacities equally critical to shape perception. Gestalt processes of figure-ground organization, grouping, and contour integration help define the "units" to which shape properties are attributed (Palmer and Rock [1994]; Feldman [2007]; Wagemans et al [2012]; Lande [2023]). Moreover, while a virtue of the Feldman/Singh model is that it predicts the perceived part structure of objects, the perception of parthood is itself a rich capacity that follows known organizational principles-e.g., minima and short-cut rules (Hoffman and Richards [1984]; Singh et al [1999]).…”