Predictive coding of action intentions in dorsal and ventral visual stream is based on visual anticipations, memory-based information and motor preparation

Abstract

Predictions of upcoming movements are based on several types of neural signals that span the visual, somatosensory, motor and cognitive system. Thus far, pre-movement signals have been investigated while participants viewed the object to be acted upon.Here, we studied the contribution of information other than vision to the classification of preparatory signals for action, even in absence of online visual information. We used functional magnetic resonance imaging (fMRI) and multivoxel pattern analysis (MVPA) to test whether the neural signals evoked by visual, memory-based and somato-motor information can be reliably used to predict upcoming actions in areas of the dorsal and ventral visual stream during the preparatory phase preceding the action, while participants were lying still. Nineteen human participants (nine women) performed one of two actions towards an object with their eyes open or closed. Despite the well-known role of ventral stream areas in visual recognition tasks and the specialization of dorsal stream areas in somato-motor processes, we decoded action intention in areas of both streams based on visual, memory-based and somato-motor signals. Interestingly, we could reliably decode action intention in absence of visual information based on neural activity evoked when visual information was available, and vice-versa. Our results show a similar visual, memory and somato-motor representation of action planning in dorsal and ventral visual stream areas that allows predicting action intention across domains, regardless of the availability of visual information.3 KeywordsFunctional magnetic resonance imaging (fMRI), multivoxel pattern analysis (MVPA), humans, actions, predictive coding, vision

Neural encoding and functional interactions underlying pantomimed movements

“…Regarding goal encoding, the possible description of this type of information might allow to draw inferences on the specific role of premotor, parietal and temporal nodes of the network. We expect to show goal encoding within regions of the IPL (SMG, aIPS), as reported in recent MVPA studies (Gallivan et al 2013b ; Chen et al 2016 , 2018 ; Turella et al 2020 ; Monaco et al 2020 ).…”

“…The parameters for data acquisition were similar to previous published work of our lab (Monaco et al 2019 , 2020 ; Turella et al 2020 ). All the MR data were acquired with a 4 T Bruker MedSpec scanner using an 8-channel head coil.…”

“…Each experimental trial followed the same structure (Fig. 1 , for a similar paradigm see Monaco et al 2019 , 2020 ). We delivered a verbal cue (duration 1 s) to the participants signaling the action to be performed.…”

Cortical areas involved in grasping and reaching actions with and without visual information: an ALE meta-analysis of neuroimaging studies

“…We included these studies in the Vision condition because the contrast did not subtract the visual processing of the grasping hand however, the inclusion of contrasts subtracting some activity related to visual processing might have reduced the sensitivity to detect activation in early visual areas. Also, while univariate analysis might lack the sensitivity to reveal activation in ventral stream areas and early visual cortex under lack of vision, recent multivoxel pattern analysis studies have shown different representations for grasping and reaching action planning with and without visual information in ventral stream and early visual areas (Monaco et al, 2019, 2021). This difference in results indicates that univariate and multivariate analysis provide complementary and not necessarily equivalent information, with MVPA being more sensitive to distributed representation of information of content, and univariate analysis showing more sensitivity to the overall engagement in a task (Coutanche, 2013; Davis et al, 2014; Jimura & Poldrack, 2012).…”

Action planning modulates the representation of object features in human fronto-parietal and occipital cortex

“…Picking up a pen, for example, would be more successful when one is focused on its orientation rather than its color. Considerable research has investigated the role of fronto-parietal reaching and grasping networks in successfully executing actions (for reviews see: Vesia and Crawford 2012;Gallivan and Culham 2015), and multivoxel pattern analysis has allowed examining the representation of action intention in fronto-parietal and temporaloccipital cortices seconds before participants start to move (Gallivan et al 2011;Gallivan, Chapman, et al 2013;Monaco et al 2019). Action planning strongly relies on the representation of our surrounding for generating accurate and effective movements, and at the same time, it enhances the detection of features that are relevant for behaviour (Gutteling et al 2011).…”