Extrastriate Visual Areas Integrate Form Features over Space and Time to Construct Representations of Stationary and Rigidly Rotating Objects

Abstract

When an object moves behind a bush, for example, its visible fragments are revealed at different times and locations across the visual field. Nonetheless, a whole moving object is perceived. Unlike traditional modal and amodal completion mechanisms known to support spatial form integration when all parts of a stimulus are simultaneously visible, relatively little is known about the neural substrates of the spatiotemporal form integration processes involved in generating coherent object representations from a succession visible fragments. We use fMRI to identify brain regions involved in two mechanisms supporting the representation of stationary and rigidly rotating objects whose form features are shown in succession: Spatiotemporal Form Integration (STFI) and Position Updating. STFI allows past and present form cues to be integrated over space and time into a coherent object even when the object is not visible in any given frame. STFI can occur whether or not the object is moving. Position updating allows us to perceive a moving object, whether rigidly rotating or translating, even when its form features are revealed at different times and locations in space. Our results suggest that STFI is mediated by visual regions beyond V1 and V2. Moreover, while widespread cortical activation has been observed for other motion percepts derived solely from form-based analyses (Krekelberg, Vatakis, & Kourtzi, 2005; Tse, 2006), increased responses for the position updating that leads to rigidly rotating object representations were only observed in visual areas KO and possibly hMT+, indicating that this is a distinct and highly specialized type of processing.

Dynamic Volume Completion and Deformation

“…Here the perceived dynamically deforming volumes can only result from the construction of a deforming volume that links behind the occluder and over time, from one visible inducer at time t1 to a later one at time t2. Future work will have to work out the dynamics of temporal integration of volume completion over discrete image segments presented at different times, but it is likely to be comparable with the temporal dynamics found for spatiotemporal integration of nonclosed surfaces ( McCarthy et al., 2015 ).…”

“…But when given a dynamic version, most people spontaneously report seeing the deforming bands, and report, again spontaneously, that they appear to be made of something elastic, like rubber, taffy or dough. This suggests that the dynamic nature of the stimuli plays a role in the switch to an interpretation consistent with volume completion, just as sequentially occurring occlusion cues can facilitate amodal and modal surface completion in the nonvolumetric surface domain ( McCarthy, Kohler, Tse, & Caplovitz, 2015 ).…”

Perceived group size is determined by the centroids of the component elements

“…It is well known, however, that not all of the objects we perceive are explicitly defined. A fundamental characteristic of our visual experience is that it is based in large part on constructive processes that can generate object representations from spatially and temporally disparate and localized visual elements ( McCarthy, Erlikhman & Caplovitz, 2017 ; McCarthy, Kohler, Tse, & Caplovitz, 2015a ; McCarthy, Strother, & Caplovitz, 2015b ). This makes it possible for our visual system to deal with ambiguities brought on by occlusion and the limitations of our visual system.…”

“…Instead, it is a series of segments that are separated by the fence. By integrating the disparate sources of information corresponding to the visible portions of the flower, the constructive processes of perceptual grouping serve an important role in enabling us to experience a stable and sufficiently accurate representation of the world around us – namely the presence of a flower with a given size and shape that is present behind the fence ( Kellman & Shipley, 1991 ; McCarthy et al, 2015a ; McCarthy et al, 2015b ; McCarthy et al, 2017 ; Palmer, Kellman, & Shipley, 2006 ).…”

On the Spatiotemporal Nature of Vision, as Revealed by Covered Bridges and Puddles: A Dispatch from Vermont

“…Retinal painting, anorthoscopic perception and amodal completion are terms to describe visual phenomena that highlight the spatiotemporal integrative mechanisms that underlie primate vision. Vision scientists have spent over a century studying the mechanisms underlying these phenomena through the use of behavioural (Fendrich & Mack, 1981;Fendrich et al, 2005;Gerbino, 2020;Helmholtz & Southall, 1867/1962Lorenceau and Shiffrar, 1992;Michotte & Burke, 1951/1962Schweitzer & Rolfs, 2020;Stanley & Molloy, 1975;Zöllner, 1862;Tyler, 2019) and more recently, neuropsychological (McCarthy et al, 2015;Orlov & Zohary, 2018;Thielen et al, 2019;Yin et al, 2002) experiments in which simplified stimuli were presented on computer monitors. Just as the value of carefully controlled experiments is undeniable, so too is the ennui of studying the boundlessly rich experiences vision offers us within the constrained confines of the testing room.…”