Network Representations of Facial and Bodily Expressions: Evidence From Multivariate Connectivity Pattern Classification

Abstract

Emotions can be perceived from both facial and bodily expressions. Our previous study has found the successful decoding of facial expressions based on the functional connectivity (FC) patterns. However, the role of the FC patterns in the recognition of bodily expressions remained unclear, and no neuroimaging studies have adequately addressed the question of whether emotions perceiving from facial and bodily expressions are processed rely upon common or different neural networks. To address this, the present study collected functional magnetic resonance imaging (fMRI) data from a block design experiment with facial and bodily expression videos as stimuli (three emotions: anger, fear, and joy), and conducted multivariate pattern classification analysis based on the estimated FC patterns. We found that in addition to the facial expressions, bodily expressions could also be successfully decoded based on the large-scale FC patterns. The emotion classification accuracies for the facial expressions were higher than that for the bodily expressions. Further contributive FC analysis showed that emotion-discriminative networks were widely distributed in both hemispheres, containing regions that ranged from primary visual areas to higher-level cognitive areas. Moreover, for a particular emotion, discriminative FCs for facial and bodily expressions were distinct. Together, our findings highlight the key role of the FC patterns in the emotion processing, indicating how large-scale FC patterns reconfigure in processing of facial and bodily expressions, and suggest the distributed neural representation for the emotion recognition. Furthermore, our results also suggest that the human brain employs separate network representations for facial and bodily expressions of the same emotions. This study provides new evidence for the network representations for emotion perception and may further our understanding of the potential mechanisms underlying body language emotion recognition.

Dual-site TMS demonstrates causal functional connectivity between the left and right posterior temporal sulci during facial expression recognition

“…Face network representations of expression recognition have been also successfully decoded from fMRI data (e.g., 38,39,40). Liang and colleagues [38] found that facial expressions are represented by the large-scale functional connectivity patterns which vary across different expressions. Consistent with our results, they found that bilateral pSTS constitutes the core component of this network, suggesting the interactive nature of the neural expression recognition across hemispheres.…”

Recognition Characteristics of Facial and Bodily Expressions: Evidence From ERPs

“…Overall, bodily and facial expressions have been found to overlap in neural processing mechanisms, and there may be interactions between them ( Zhu and Luo, 2012 ; Hietanen et al, 2014 ; Borgomaneri et al, 2015 ; Borhani et al, 2016 ). However, Liang et al (2019) used fMRI to explore the network representation of facial and bodily expressions and found that the human brain employs separate network representations for facial and bodily expressions of the same emotions. In this study, we expand on the two emotions of happiness and fear examined by Gu et al (2013) to explore whether the identification of positive and negative emotions is consistent with the three-stage model and the integration characteristics of facial and bodily expressions.…”

STN: a new tensor network method to identify stimulus category from brain activity pattern

“…( 19) In order to test the effect of the model, this paper adopt two real collected data sets, one was MEG data, which was collected about four types of object pictures, the other was fMRI data, which was collected about four kinds of emotional face pictures. At present, these two modal of neuroimaging data sets have been proved by many studies that they can construct the stable brain network pattern [24], [25].…”