Decoding Multivoxel Representations of Affective Scenes in Retinotopic Visual Cortex

Abstract

The perception of opportunities and threats in complex scenes represents one of the main functions of the human visual system. In the laboratory, its neurophysiological basis is often studied by having observers view pictures varying in affective content. This body of work has consistently shown that viewing emotionally engaging, compared to neutral, pictures (1) heightens blood flow in limbic structures and frontoparietal cortex, as well as in anterior ventral and dorsal visual cortex, and (2) prompts an increase in the late positive event-related potential (LPP), a scalp-recorded and time-sensitive index of engagement within the network of aforementioned neural structures. The role of retinotopic visual cortex in this process has, however, been contentious, with competing theoretical notions predicting the presence versus absence of emotion-specific signals in retinotopic visual areas. The present study used multimodal neuroimaging and machine learning to address this question by examining the large-scale neural representations of affective pictures. Recording EEG and fMRI simultaneously while observers viewed pleasant, unpleasant, and neutral affective pictures, and applying multivariate pattern analysis to single-trial BOLD activities in retinotopic visual cortex, we identified three robust findings: First, unpleasant-versus-neutral decoding accuracy, as well as pleasant-versus-neutral decoding accuracy, were well above chance level in all retinotopic visual areas, including primary visual cortex. Second, the decoding accuracy in ventral visual cortex, but not in early visual cortex or dorsal visual cortex, was significantly correlated with LPP amplitude. Third, effective connectivity from amygdala to ventral visual cortex predicted unpleasant-versus-neutral decoding accuracy, and effective connectivity from ventral frontal cortex to ventral visual cortex predicted pleasant-versus-neutral decoding accuracy. These results suggest that affective pictures evoked valence-specific multivoxel neural representations in retinotopic visual cortex and that these multivoxel representations were influenced by reentry signals from limbic and frontal brain regions.

Decoding Neural Representations of Affective Scenes in Retinotopic Visual Cortex

“…Where do affective signals in retinotopic visual cortex come from? One recent hypothesis stresses that visual cortex can innately discriminates stimuli varying in affective significance (see Miskovic & Anderson, 2018 for a review). For example, both animal model studies and human studies (Li et al, 2019;Weinberger, 2004;Thigpen et al, 2017) have found that sensory cortex is able to encode the threat content of a visual stimulus, typically after extensive learning.…”

“…A second group of hypotheses states that sensory cortex, including retinotopic areas, may itself code for emotional qualities of a stimulus, without the necessity for recurrent processing (see Miskovic & Anderson, 2018, for a review). Evidence supporting this hypothesis comes from empirical studies in experimental animals (Li et al, 2019;Weinberger, 2004) and human observers (Thigpen et al, 2017), in which the extensive pairing of simple sensory cues such as tilted lines or sinusoidal gratings with emotionally relevant outcomes shapes early sensory responses (Miskovic & Keil, 2012).…”