Object Vision in a Structured World

Abstract

In natural vision, objects appear at typical locations, both with respect to visual space (e.g., an airplane in the upper part of a scene) and other objects (e.g., a lamp above a table ). Recent studies have shown that object vision is strongly adapted to such positional regularities. In this review we synthesize these developments, highlighting that adaptations to positional regularities facilitate object detection and recognition, and sharpen the representations of objects in visual cortex. These effects are pervasive across various types of high-level content. We posit that adaptations to real-world structure collectively support optimal usage of limited cortical processing resources. Taking positional regularities into account will thus be essential for understanding efficient object vision in the real world. Positional Regularities in Object VisionMany natural behaviors crucially depend on accurately perceiving objects in the environment. Consequently, understanding object vision has been a core endeavor in cognitive neuroscience for many years, and recent decades have yielded exciting insights into how the human visual system processes various types of objects [1][2][3][4][5]. By and large, these insights have come from studies investigating the processing of individual objects presented at arbitrary locations (usually at fixation). However, in natural vision many objects often appear in specific locations both with respect to visual space (e.g., airplanes in the sky) and relative to other objects (e.g., lamps above tables).Although it has already been well established that such real-world positional regularities furnish observers with cognitive strategies that support effective behaviors (e.g., by providing schemata for economical memory storage [6][7][8] and efficient attentional allocation during search [9-11]), more recent work has begun to investigate the influence of real-world structure on how we perceive and represent objects. A rapidly burgeoning literature now indicates that positional regularities affect basic perceptual analysis both in terms of neural responses in visual cortex (e.g., by shaping tuning properties of object-selective regions) and perceptual sensitivity in psychophysical tasks (e.g., by facilitating object recognition and detection). Intriguingly, the general relevance of these effects has now been demonstrated across a range of high-level visual domains, including everyday objects, faces and bodies, words, and even social interactions between people. Drawing from both the neuroimaging and behavioral literatures, in this review we synthesize recent findings across processing levels and visual domains, and discuss how their resulting insights improve our understanding of real-world object vision.

Cortical sensitivity to natural scene structure

“…Focusing on the interplay of multiple scene elements, the current study shows that on higher levels of the scene processing hierarchy, the visual system uses spatial regularities to concurrently process the multiple elements of complex scenes in an efficient way. This result is in line with the emerging view that real-world structure facilitates processing in the visual system across diverse naturalistic contents (Kaiser, Quek, Cichy, & Peelen, 2019).…”

“…A major reason for this perceptual efficiency lies in the structure of natural scenes: for instance, a scene's spatial structure tells us where specific objects can be found and its categorical structure tells us which objects are typically encountered within the scene (Kaiser, Quek, Cichy, & Peelen, 2019;Oliva & Torralba, 2007;Võ, Boettcher, & Draschkow, 2019;Wolfe, Võ, Evans, & Greene, 2011).…”

Real-world structure facilitates the rapid emergence of scene category information in visual brain signals

“…The early facilitation of category information is consistent with results from single-object processing, where representations of individual objects are rapidly enhanced -within the first 150ms of vision -when the objects appear in their typical real-world locations, such as an eye in the upper visual field (Issa & DiCarlo, 2012) or a shoe in the lower visual field . Together, these findings therefore support the idea that real-world structure can boost basic visual analysis across diverse stimuli and processing levels (Kaiser et al, 2019a).…”

“…In everyday situations, the input to our visual system is not random. It rather arises from structured scenes, in which information follows meaningful distributions across space (Bar 2004;Kaiser et al, 2019a;Oliva & Torralba, 2007;Võ et al, 2019;Wolfe et al, 2011). Neuroimaging studies have shown that the visual system is sensitive to this structure, with cortical responses differing when scene elements do or do not adhere to typical real-world structure (Abassi & Papeo, 2019;Baldassano et al, 2017;Bilalic et al, 2019;Kaiser et al, 2014;2019b;Kaiser & Peelen, 2018;Kim & Biederman, 2011;Roberts & Humphreys, 2010).…”

Object representations in the human brain reflect the co-occurrence statistics of vision and language

“…One open question is whether visual cortex contains representations that reflect the natural statistics of object locations in images. Here we examined object co-occurrence independent of object location, but it is known that the visual system also takes advantage of associations between objects and spatial locations 60 and that perceptual behaviors and cortical representations are sensitive to violations of the typical positions of objects 2,61-63 . Future investigations could utilize annotated image databases to explicitly quantify these spatial statistics and to examine how they relate to the object representations of visual cortex.…”