Visual information representation and rapid-scene categorization are simultaneous across cortex: An MEG study

Abstract

Perceiving the visual world around us requires the brain to represent the features of stimuli and to categorize the stimulus based on these features. Incorrect categorization can result either from errors in visual representation or from errors in processes that lead to categorical choice. To understand the temporal relationship between the neural signatures of such systematic errors, we recorded whole-scalp magnetoencephalography (MEG) data from human subjects performing a rapid-scene categorization task. We built scene category decoders based on (1) spatiotemporally resolved neural activity, (2) spatial envelope (SpEn) image features, and (3) behavioral responses. Using confusion matrices, we tracked how well the pattern of errors from neural decoders could be explained by SpEn decoders and behavioral errors, over time and across cortical areas. Across the visual cortex and the medial temporal lobe, we found that both SpEn and behavioral errors explained unique variance in the errors of neural decoders. Critically, these effects were nearly simultaneous, and most prominent between 100 and 250ms after stimulus onset. Thus, during rapid-scene categorization, neural processes that ultimately result in behavioral categorization are simultaneous and co-localized with neural processes underlying visual information representation.

Disentangling the Independent Contributions of Visual and Conceptual Features to the Spatiotemporal Dynamics of Scene Categorization

“…Our decoding results revealed that decodable scene category information peaked between 150 and 200 ms after image onset and persisted across the trial epoch. These values are consistent with previous M/EEG studies of object-and scene categorization (Bankson, Hebart, Groen, & Baker, 2018;Carlson, Tovar, Alink, & Kriegeskorte, 2013;Cichy, Pantazis, & Oliva, 2014;Clarke, Taylor, Devereux, Randall, & Tyler, 2013;Ramkumar et al, 2016). While earlier decoding has been reported for image exemplars (~100 ms, (Carlson et al, 2013;Cichy et al, 2014)), it has remained unclear whether this performance reflects image identity per se, or the lower-level visual features that are associated with that exemplar.…”

“…While this generally held true for the nine models used here, it should be noted that the gist features (Oliva & Torralba, 2001) were an exception (see Figure 1). Therefore, we have refrained from strongly interpreting results for that model, particularly the observation that this feature was not significantly predictive of the behavioral RDM (see Table 1) given previous reports that gist features can strongly influence categorization behavior (M. ), vERPs (Hansen, Noesen, Nador, & Harel, 2018), MEG patterns (Ramkumar et al, 2016), and fMRI activation patterns (Watson et al, 2017(Watson et al, , 2014.…”

“…The number of orientations varies with spatial frequency, with 8 orientations at the highest frequency, 6 in the mid-range, and 4 at the lowest, for a total of 1152 features (64 windows x (8+6+4) orientations per scale). These features have been shown to explain significant variance in fMRI and MEG responses throughout visual cortex (Ramkumar et al, 2016;Watson, Andrews, & Hartley, 2017;Watson, Hartley, & Andrews, 2014). Given their similarity to the log-Gabor filters described above, following (Oliva & Torralba, 2001), we used linear discriminant analysis (LDA) to learn weights on the filter outputs that were optimized for classifying the 30 scene categories of this database.…”

“…Human scene processing is characterized by its high speed: not only do observers require little viewing time to reliably understand scene content (M. R. Potter, Wyble, Hagmann, & McCourt, 2014), but scene-specific neural responses have also been observed less than 200 ms after scene presentation (Bastin et al, 2013;Ramkumar, Hansen, Pannasch, & Loschky, 2016;Thorpe, Fize, & Marlot, 1996). However, we know comparatively little about the processing stages that transform the retinal image into a semantically rich categorical representation.…”

Spatiotemporal dynamics in human visual cortex rapidly encode the emotional content of faces

“…While these results point to relatively late effects, high-level categorization has been shown to occur at shorter latencies in the visual system of primates, especially when tested using sensitive multivariate methods (Cauchoix et al, 2016). In humans, multivariate pattern analysis (MVPA) of non-invasive electrophysiological data has shown potential to achieve a similar level of sensitivity, demonstrating rapid categorization along the ventral stream (Cauchoix, Barragan-Jason, Serre, & Barbeau, 2014;Isik, Meyers, Leibo, & Poggio, 2014;Ramkumar, Hansen, Pannasch, & Loschky, 2016). Fast decoding of object category was achieved at 100 ms from small neuronal populations in primates (Hung & Poggio, 2005) and from invasively recorded responses in human visual cortex (Li & Lu, 2009).…”