No evidence from MVPA for different processes underlying the N300 and N400 incongruity effects in object-scene processing

Abstract

Attributing meaning to diverse visual input is a core feature of human cognition. Violating environmental expectations (e.g., a toothbrush in the fridge) induces a late event-related negativity of the event-related potential/ERP. This N400 ERP has not only been linked to the semantic processing of language, but also to objects and scenes. Inconsistent object-scene relationships are additionally associated with an earlier negative deflection of the EEG signal between 250 and 350 ms. This N300 is hypothesized to reflect pre-semantic perceptual processes. To investigate whether these two components are truly separable or if the early object-scene integration activity (250–350 ms) shares certain levels of processing with the late neural correlates of meaning processing (350–500 ms), we used time-resolved multivariate pattern analysis (MVPA) where a classifier trained at one time point in a trial (e.g., during the N300 time window) is tested at every other time point (i.e., including the N400 time window). Forty participants were presented with semantic inconsistencies, in which an object was inconsistent with a scene's meaning. Replicating previous findings, our manipulation produced significant N300 and N400 deflections. MVPA revealed above chance decoding performance for classifiers trained during time points of the N300 component and tested during later time points of the N400, and vice versa. This provides no evidence for the activation of two separable neurocognitive processes following the violation of context-dependent predictions in visual scene perception. Our data supports the early appearance of high-level, context-sensitive processes in visual cognition.

Cortical sensitivity to natural scene structure

“…Interestingly, after 250 ms brain responses not only become sensitive to scene structure, but also to object-scene consistencies (Draschkow et al, 2018;Ganis & Kutas, 2003;Mudrik et al, 2010;Võ & Wolfe, 2013). Together, these results suggest a dedicated processing stage for the structural analysis of objects, scenes, and their relationships, which is different from basic perceptual processing.…”

“…Radoslaw M. Cichy https://orcid.org/0000-0003-4190-6071 ENDNOTES 1 Related studies on object-object and object-scene consistencies typically yield large effect sizes which exceed this value, both for fMRI responses, d = 0.72 (Brandman & Peelen, 2017), d = 0.67 (Kaiser & Peelen, 2018), d = 2.14 (Kim & Biederman, 2011), d = 0.94 (Roberts & Humphreys, 2010), and EEG responses, d = 0.71 (Draschkow, Heikel, Võ, Fiebach, & Sassenhagen, 2018), d = 0.88 (Ganis & Kutas, 2003), d = 0.67 (Mudrik, Lamy, & Deouell, 2010), d = 0.69 (Võ & Wolfe, 2013).…”

Disentangling the Independent Contributions of Visual and Conceptual Features to the Spatiotemporal Dynamics of Scene Categorization

“…Indeed, violations of scene-object context have been observed in the 250-350 ms post-image window (Mudrik, Shalgi, Lamy, & Deouell, 2014), as well as in the classical N400 window (Ganis & Kutas, 2003;Võ & Wolfe, 2013). However, recent decoding results have shown that these two windows contain similar image information (Draschkow, Heikel, Võ, Fiebach, & Sassenhagen, 2018) and may be reflecting similar neural processes. It is worth noting that the time course of these ERPs reflects an upper bound to the time course of semantic processing, and that the current encoding and decoding techniques may reveal the processes themselves while the ERPs reflect the outcomes.…”

Your Brain on Comics: A Cognitive Model of Visual Narrative Comprehension

“…Incongruous aspects of scenes also modulate the N400, such as when unexpected objects appear within a scene, like soccer players kicking a roll of toilet paper instead of a ball (Ganis & Kutas, ; Sauvé, Harmand, Vanni, & Brodeur, ; Võ & Wolfe, ). This N400 to images may also follow an N300 , an additional frontal negativity peaking around 300 ms which has been taken to reflect the semantic identification or categorization of visual objects (Draschkow, Heikel, Võ, Fiebach, & Sassenhagen, ; Hamm, Johnson, & Kirk, ; McPherson & Holcomb, ).…”