Parts and Wholes in Scene Processing

Abstract

During natural vision, our brains are constantly exposed to complex, but regularly structured environments. Real-world scenes are defined by typical part–whole relationships, where the meaning of the whole scene emerges from configurations of localized information present in individual parts of the scene. Such typical part–whole relationships suggest that information from individual scene parts is not processed independently, but that there are mutual influences between the parts and the whole during scene analysis. Here, we review recent research that used a straightforward, but effective approach to study such mutual influences: By dissecting scenes into multiple arbitrary pieces, these studies provide new insights into how the processing of whole scenes is shaped by their constituent parts and, conversely, how the processing of individual parts is determined by their role within the whole scene. We highlight three facets of this research: First, we discuss studies demonstrating that the spatial configuration of multiple scene parts has a profound impact on the neural processing of the whole scene. Second, we review work showing that cortical responses to individual scene parts are shaped by the context in which these parts typically appear within the environment. Third, we discuss studies demonstrating that missing scene parts are interpolated from the surrounding scene context. Bridging these findings, we argue that efficient scene processing relies on an active use of the scene's part–whole structure, where the visual brain matches scene inputs with internal models of what the world should look like.

Scene-selective regions encode the vertical position of navigationally relevant information in young and older adulthood

“…These results resonate with previous work unveiling similar neural representations in the visual system for parts of scenes that occupy identical locations along the vertical axis 74,75 . We add further compelling empirical support to the hypothesis formulated by Kaiser and Cichy (2021) 76 that the human brain is wired to sort visual inputs according to their upper or lower positions. An interesting question resides in understanding why verticality is an important cue for scene processing and spatial navigation.…”

Alpha-frequency feedback to early visual cortex orchestrates coherent naturalistic vision

“…These regions exhibited stronger representations for spatiotemporally coherent stimuli placed in the two hemifields, despite a lack of sensitivity to the ipsilateral input alone. Scene-selective cortex is sensitive to the typical spatial configuration of scene stimuli 18,19,35,36 , allowing it to create feedback signals that carry information about whether and how stimuli need to be integrated at lower levels of the visual hierarchy. To further support this notion, future studies should explicitly map cortico-cortical connectivity during natural perception.…”

Alpha-frequency feedback to early visual cortex orchestrates coherent naturalistic vision

“…We reasoned that regions capable of exerting integration-related feedback should show stronger representations of spatiotemporally coherent inputs that can be integrated, compared to incoherent inputs that do not. Scene-selective areas in visual cortex are a strong contender for the source of this feedback, as they have been previously linked to the spatial integration of coherent scene information ( 18 , 19 ).…”