Classification of emotion categories based on functional connectivity patterns of the human brain

Saarimäki, Heini; Glerean, Enrico; Смирнов, Д. А.; Mynttinen, Henri; Jä̈askëlainen, Iiro P.; Sams, Mikko; Nummenmaa, Lauri

doi:10.1016/j.neuroimage.2021.118800

Cited by 30 publications

(29 citation statements)

References 101 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although these studies demonstrated the feasibility to decode specific emotions from FC patterns, the performance varied between emotional categories, in particular the specificity and accuracy for sadness remained limited. For instance, despite a high accuracy obtained for most emotions in Saarimä ki et al's study (Saarimä ki et al, 2022), it revealed a classification accuracy of sadness was close to chance level (18% accuracy). One possible reason might be the different time frames of emotional experiences and thus the difficulty in robustly evoking strong and engaging feelings of sadness with experimental stimuli as short as one minute.…”

Section: Introductionmentioning

confidence: 69%

“…For example, in Saarimä ki et al's work(Saarimä ki et al, 2022), the mean classification accuracy on sadness was the lowest (18%), which was close to the chance level (16.67%). A similar situation occurred in another fMRI-based emotion recognition work(Saarimä ki et al, 2016), where the emotion prediction result on sadness was still the lowest.…”

mentioning

confidence: 72%

“…It has been further demonstrated the feasibility of analyzing FC at the level of concordant patterns of temporal variations under the mental states during rest and task (Cohen and D'Esposito, 2016). Recently, a number of FC-based emotion-related brain network studies have been conducted (Pessoa, 2017(Pessoa, , 2018Putkinen et al, 2021;Saarimä ki et al, 2022). For example, Putkinen et al examined the emotion-related functional neural basis of four separate emotions evoked by music (happiness, sadness, fear, and tender) and observed that brain activity patterns in the auditory and primary motor cortices correlated with the respective emotional states (Putkinen et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

“…Saarimä ki et al introduced multivariate pattern analysis to develop a crosssubject emotion recognition approach that is based on whole-brain FC profiles. The study collected brain activity in 16 subjects during fMRI while the subjects were presented with 1-minute emotional audio narratives from six emotion categories (anger, fear, disgust, happiness, sadness, and surprise) (Saarimä ki et al, 2022). The results showed that the most accurate emotion prediction could be obtained from the Default Mode Network (DMN), indicating an important contribution of the DMN in emotional processing under naturalistic conditions.…”

Section: Introductionmentioning

confidence: 99%

“…Human emotion represents a dynamic process involving different levels of processing and integration (Cowen and Keltner, 2017; Horikawa et al, 2020). Determining the specific neurophysiological basis of emotions and their distinct neural representations can facilitate discriminating specific emotional states from other mental processes (Putkinen et al, 2021; Saarimäki et al, 2022; Vytal and Hamann, 2010; Zhou et al, 2020; Zhou et al, 2021) and help evaluate emotion-specific dysregulations and corresponding treatment approaches (Reddan et al, 2018; Xu et al, 2022; Zhang et al, 2022). To capture the complex and dynamic nature of emotional processes, an increasing number of functional Magnetic Resonance Imaging (fMRI) studies examined cortical and subcortical responses during naturalistic contexts such as narratives or movies (Jääskeläinen et al, 2021; Putkinen et al, 2021; Saarimäki et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Whole-Brain fMRI Functional Connectivity Signatures Predict Sustained Emotional Experience in Naturalistic Contexts

Zhang

et al. 2022

Preprint

View full text Add to dashboard Cite

Determining and decoding emotional brain processes under ecologically valid conditions remains a key challenge in affective neuroscience. The current functional magnetic resonance imaging (fMRI) based emotion decoding studies are mainly based on brief and isolated episodes of emotion induction, while sustained emotional experience in naturalistic environments that mirror daily life experiences are scarce. Here we use 10-minute movie clips as ecologically valid emotion-evoking procedures in n=52 individuals to explore emotion-specific fMRI functional connectivity (FC) profiles on the whole-brain level at high spatial resolution (400 atlas based parcels). Employing machine-learning based decoding and cross validation procedures allowed to develop predictive FC profiles that can accurately distinguish sustained happiness and sadness and that generalize across movies and subjects. Both functional brain network-based and subnetwork-based emotion prediction results suggest that emotion manifests as distributed representation of multiple networks, rather than a single functional network or subnetwork. Further, the results show that the Visual Network (VN) and Default Mode Network (DMN) associated functional networks, especially VN-DMN, exhibit a strong contribution to emotion prediction. To further estimate the cumulative effect of naturalistic long-term movie-based video-evoking emotions, we divide the 10-min episode into three stages: early stimulation (1 ~ 200 s), middle stimulation (201 ~ 400 s), and late stimulation (401 ~ 600 s) and examine the emotion prediction performance at different stimulation stages. We found that the late stimulation has a stronger predictive ability (accuracy=85.32%, F1-score=85.62%) compared to early and middle stimulation stages, implying that continuous exposure to emotional stimulation can lead to more intense emotions and further enhance emotion-specific distinguishable representations. The present work demonstrates that sustained sadness and happiness under naturalistic conditions are presented in emotion-specific network profiles and these expressions may play different roles in the generation and modulation of emotions. These findings elucidate the importance of network level adaptations for sustained emotional experiences during naturalistic contexts and open new venues for imaging network level contributions under naturalistic conditions.

show abstract

Section: Introductionmentioning

confidence: 69%

mentioning

confidence: 72%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Whole-Brain fMRI Functional Connectivity Signatures Predict Sustained Emotional Experience in Naturalistic Contexts

Zhang

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Aberrant degree centrality profiles during rumination in major depressive disorder

Jia,

Chen,

et al. 2023

Human Brain Mapping

View full text Add to dashboard Cite

Rumination is closely linked to the onset and maintenance of major depressive disorder (MDD). Prior neuroimaging studies have identified the association between self‐reported rumination trait and the functional coupling among a network of brain regions using resting‐state functional magnetic resonance imaging (MRI). However, little is known about the underlying neural circuitry mechanism during active rumination in MDD. Degree centrality (DC) is a simple metric to denote network integration, which is critical for higher‐order psychological processes such as rumination. During an MRI scan, individuals with MDD (N = 45) and healthy controls (HC, N = 46) completed a rumination state task. We examined the interaction effect between the group (MDD vs. HC) and condition (rumination vs. distraction) on vertex‐wise DC. We further characterized the identified brain region's functional involvement with Neurosynth and BrainMap. Network‐wise seed‐based functional connectivity (FC) analysis was also conducted for the identified region of interest. Finally, exploratory correlation analysis was conducted between the identified region of interest's network FCs and self‐reported in‐scanner affect levels. We found that a left superior frontal gyrus (SFG) region, generally overlapped with the frontal eye field, showed a significant interaction effect. Further analysis revealed its involvement with executive functions. FCs between this region, the frontoparietal, and the dorsal attention network (DAN) also showed significant interaction effects. Furthermore, its FC to DAN during distraction showed a marginally significant negative association with in‐scanner affect level at the baseline. Our results implicated an essential role of the left SFG in the rumination's underlying neural circuitry mechanism in MDD and provided novel evidence for the conceptualization of rumination in terms of impaired executive control.

show abstract

Multimodal emotion recognition: A comprehensive review, trends, and challenges

Ramaswamy,

Palaniswamy

2024

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Automatic emotion recognition is a burgeoning field of research and has its roots in psychology and cognitive science. This article comprehensively reviews multimodal emotion recognition, covering various aspects such as emotion theories, discrete and dimensional models, emotional response systems, datasets, and current trends. This article reviewed 179 multimodal emotion recognition literature papers from 2017 to 2023 to reflect on the current trends in multimodal affective computing. This article covers various modalities used in emotion recognition based on the emotional response system under four categories: subjective experience comprising text and self‐report; peripheral physiology comprising electrodermal, cardiovascular, facial muscle, and respiration activity; central physiology comprising EEG, neuroimaging, and EOG; behavior comprising facial, vocal, whole‐body behavior, and observer ratings. This review summarizes the measures and behavior of each modality under various emotional states. This article provides an extensive list of multimodal datasets and their unique characteristics. The recent advances in multimodal emotion recognition are grouped based on the research focus areas such as emotion elicitation strategy, data collection and handling, the impact of culture and modality on multimodal emotion recognition systems, feature extraction, feature selection, alignment of signals across the modalities, and fusion strategies. The recent multimodal fusion strategies are detailed in this article, as extracting shared representations of different modalities, removing redundant features from different modalities, and learning critical features from each modality are crucial for multimodal emotion recognition. This article summarizes the strengths and weaknesses of multimodal emotion recognition based on the review outcome, along with challenges and future work in multimodal emotion recognition. This article aims to serve as a lucid introduction, covering all aspects of multimodal emotion recognition for novices.This article is categorized under: Fundamental Concepts of Data and Knowledge > Human Centricity and User Interaction Technologies > Cognitive Computing Technologies > Artificial Intelligence

show abstract

Classification of emotion categories based on functional connectivity patterns of the human brain

Cited by 30 publications

References 101 publications

Whole-Brain fMRI Functional Connectivity Signatures Predict Sustained Emotional Experience in Naturalistic Contexts

Whole-Brain fMRI Functional Connectivity Signatures Predict Sustained Emotional Experience in Naturalistic Contexts

Aberrant degree centrality profiles during rumination in major depressive disorder

Multimodal emotion recognition: A comprehensive review, trends, and challenges

Contact Info

Product

Resources

About