MEG sensor patterns reflect perceptual but not categorical similarity of animate and inanimate objects

Abstract

High-level visual cortex shows a distinction between animate and inanimate objects, as revealed by fMRI. Recent studies have shown that object animacy can similarly be decoded from MEG sensor patterns. What object properties drive this decoding? Here, we disentangled the influence of perceptual and categorical properties by presenting perceptually matched objects that were easily recognizable as being animate or inanimate (e.g., snake and rope). In a series of behavioral experiments, three aspects of perceptual dissimilarity of these objects were quantified: overall dissimilarity, outline dissimilarity, and texture dissimilarity. Neural dissimilarity of MEG sensor patterns, collected in male and female human participants, was modeled using regression analysis, in which perceptual dissimilarity (taken from the behavioral experiments) and categorical dissimilarity served as predictors of neural dissimilarity. We found that perceptual dissimilarity was strongly reflected in MEG sensor patterns from 80 ms after stimulus onset, with separable contributions of outline and texture dissimilarity. Surprisingly, MEG patterns did not distinguish between animate and inanimate objects after controlling for perceptual dissimilarity. Nearly identical results were found in a second MEG experiment that required object recognition. These results indicate that MEG sensor patterns do not capture object animacy independently of perceptual differences between animate and inanimate objects. This is in contrast to results observed in fMRI using the same stimuli, task, and analysis approach, with fMRI showing a highly reliable categorical distinction in ventral temporal cortex even when controlling for perceptual dissimilarity. The discrepancy between MEG and fMRI precludes the straightforward integration of these imaging modalities. Significance statementRecent studies have shown that multivariate analysis of MEG sensor patterns allows for a detailed characterization of the time course of visual object processing, demonstrating that the neural representational space starts to become organized by object category (e.g., separating animate and inanimate objects) from around 150 ms after stimulus onset. It is unclear, however, whether this organization truly reflects a categorical distinction or whether it reflects uncontrolled differences in perceptual similarity (e.g., most animals have four legs). Here we find that MEG sensor patterns no longer distinguish between animate and inanimate objects when controlling for perceptual differences between objects (e.g., when comparing snake and rope). These results indicate that MEG sensor patterns are primarily sensitive to visual object properties.

The representational dynamics of visual objects in rapid serial visual processing streams

“…In the current study, we characterised the representational dynamics of a large number of images in fast presentation sequences. Previous work has used MEG and EEG decoding to investigate representations of much smaller image sets using slow image presentation paradigms (Carlson et al, 2013;Cichy et al, 2014;Contini et al, 2017;Grootswagers, Ritchie, et al, 2017;Kaiser et al, 2016;Kaneshiro et al, 2015;Proklova et al, 2017;Ritchie, Tovar, & Carlson, 2015;Simanova, van Gerven, Oostenveld, & Hagoort, 2010); here we extend this work by looking at the representations of 200 objects during RSVP using standard 64-channel EEG. For 5Hz and 20Hz sequences, all 200 images could be decoded at four different categorical levels.…”

“…The current stimulus set consisted of segmented coloured objects, which were not matched on low-level features such as colour, orientation, shape, and size. Future work can build on the current paradigm using a stimulus set that for example contains orthogonal shape and category dimensions (Bracci, Kalfas, & Op de Beeck, 2017;Bracci & Op de Beeck, 2016;Proklova et al, 2017Proklova et al, , 2016, or test the decodability of these features using for example texture stimuli with similar features (Long, Konkle, Cohen, & Alvarez, 2016;Long, Yu, & Konkle, 2017). Such extensions can help unravel the relationship between object features and categories, and increase our understanding of how this inherent relationship guides categorical abstraction in the visual system.…”

“…The small correlations between the categorical models and the low-level feature models suggests that there was little overlap between the low-level features and categorical organisations in the stimulus set. To quantify the unique contributions of all models to the neural dissimilarities, we modelled the time-varying neural RDMs of each subject as a linear combination of the candidate models using a GLM (Oosterhof et al, 2016;Proklova, Kaiser, & Peelen, 2017); for each time point, the lower triangles of the neural RDM and candidate models were vectorised, and regression coefficients were obtained for all candidate models. This resulted in one beta estimate for each model, subject, and time point.…”

Temporal dynamics of the neural representation of hue and luminance polarity

“…In object perception, the decoding time course reflects the perceptual and categorical dissimilarity of stimuli, with more perceptually dissimilar stimuli (i.e. higher levels of abstraction) decodable later, and associated with computations performed by areas further along the visual-processing hierarchy (Proklova et al, 2019) (Carlson et al, 2013) (Cichy et al, 2014) (an example of increasing category abstraction is Dobermandoganimalanimate). One way of thinking about the relatively later decoding of hue, then, is that (1) hue discrimination involves greater perceptual dissimilarity (or greater category abstraction) than does luminance-contrast; and (2) it is computed either by circuits downstream of those that compute luminance contrast or requires more recurrent processing.…”

Recent advances in understanding object recognition in the human brain: deep neural networks, temporal dynamics, and context

“…snake and rope) across category in order to examine the influence of perceptual and categorical similarity on object representations. Even though the studies used identical stimuli, the results were different between the two neuroimaging modalities: they found more evidence for categorical similarity with fMRI 68 and perceptual similarity with MEG 69 .…”