Graspability Modulates the Stronger Neural Signature of Motor Preparation for Real Objects vs. Pictures

Abstract

The cognitive and neural bases of visual perception are typically studied using pictures rather than real-world stimuli. Unlike pictures, real objects are actionable solids that can be manipulated with the hands. Recent evidence from human brain imaging suggests that neural responses to real objects differ from responses to pictures; however, little is known about the neural mechanisms that drive these differences. Here, we tested whether brain responses to real objects versus pictures are differentially modulated by the “in-the-moment” graspability of the stimulus. In human dorsal cortex, electroencephalography responses show a “real object advantage” in the strength and duration of mu (μ) and low beta (β) rhythm desynchronization—well-known neural signatures of visuomotor action planning. We compared desynchronization for real tools versus closely matched pictures of the same objects, when the stimuli were positioned unoccluded versus behind a large transparent barrier that prevented immediate access to the stimuli. We found that, without the barrier in place, real objects elicited stronger μ and β desynchronization compared to pictures, both during stimulus presentation and after stimulus offset, replicating previous findings. Critically, however, with the barrier in place, this real object advantage was attenuated during the period of stimulus presentation, whereas the amplification in later periods remained. These results suggest that the “real object advantage” is driven initially by immediate actionability, whereas later differences perhaps reflect other, more inherent properties of real objects. The findings showcase how the use of richer multidimensional stimuli can provide a more complete and ecologically valid understanding of object vision.

Human memory for real-world solid objects is not predicted by responses to image displays.

“…After all, real objects are highly multidimensional; they have a definite weight, surface texture, taste, smell, and compliance, unlike images which do not have these properties. In the EEG study by Fairchild et al (2021), a barrier attenuated μ desynchronization differences between formats early after stimulus onset, but not later in the trial after stimulus offset. This led the authors to speculate that immediate effects of realness are driven by in-the-moment actionability, while more slowly evolving effects are driven by inherent properties of real objects such as distance or physical size (which remained stable in their study, irrespective of the barrier).…”

“…When observers look at (but do not touch) everyday tools, real-world exemplars elicit stronger desynchronization of the mu (μ) rhythm—a neural signature of automatic motor preparation to act—than do matched 2-D photos of the same tools (Marini et al, 2019). However, with a transparent barrier in place, the difference in μ desynchronization across formats is sharply attenuated (Fairchild et al, 2021).…”

“…Our findings address which of the differences between real objects and pictures drive the quantitative effects on performance. Numerous studies have revealed differences in behavioral (Bushong et al, 2010;Chainay & Humphreys, 2001;Gerhard et al, 2016Gerhard et al, , 2021Gomez et al, 2018;Holler et al, 2019Holler et al, , 2020Holmes & Heath, 2013;Romero et al, 2017;Sensoy et al, 2021) and brain (Fairchild et al, 2021;Freud et al, 2017;Marini et al, 2019;Snow et al, 2011) responses to real versus artificial stimuli, yet there is intense interest in determining whether these effects are explained by differences between the stimuli in visual depth information or differences related to inherent tangibility (Snow & Culham, 2021). We consistently found that real objects were more memorable than images of objects, both when the images were presented with binocular disparity, and when the real objects were viewed monocularly-indicating that that the percept of threedimensionality provided by stereopsis is not sufficient to boost memory for images (relative to real objects), nor to eliminate the memory advantage for real objects (relative to planar images of objects).…”

Reach corrections toward moving objects are faster than reach corrections toward jumping targets

“…As previously mentioned, the discrepancy between the effect of jumping and moving targets in our paradigm compared to previous results could related to our use of physical targets rather than the virtual targets used in previous studies. Physical objects elicit different neural activity than virtual objects even when these objects are merely presented to the field of view (Fairchild et al, 2021; Gallivan et al, 2009). Further, placement of physical objects within the workspace, or “graspable” objects, (Gomez et al, 2018) appears to additionally modulate brain activity, suggesting that responses we see in the present study could take advantage of this specialization of the visuomotor system.…”

“…In fact, a growing body of literature indicates that interacting with virtual representations of objects is often not a good proxy for interaction with the physical objects themselves (Snow & Culham, 2021). For example, neural activity is modulated according to whether or not objects are physical (Freud et al, 2018) and even whether or not a physical object is close enough to the body for interaction (Fairchild et al, 2021; Gallivan et al, 2009). Additionally, when planning interactions with real objects compared to virtual targets or unreachable objects, the surrounding environment is taken into account to a greater extent, potentially indicating that physical interaction is planned to a greater extent, if the object is actually reachable (Gomez et al, 2018).…”