Intersession Instability in fNIRS-Based Emotion Recognition

Abstract

Emotion recognition based on neural signals is a promising technique for the detection of patients' emotions for enhancing healthcare. However, emotion-related neural signals, such as from functional near infrared spectroscopy (fNIRS), can be affected by various psychophysiological and environmental factors. There is a paucity of literature regarding data instability and classification instability in fNIRS-based emotion recognition systems, phenomenon which may lead to user dissatisfaction and abandonment. We collected data in an fNIRS-based 2-class emotion recognition test-retest experiment (3 week interval) with visual stimuli emotion induction to examine data instability and its impact on classification accuracy. We found a 22.2% average deterioration of emotion classification accuracy between the two sessions, suggesting that classification instability is a serious problem. We found that the changes in the distributions of the selected neural signal features, as evaluated by Kullback-Leibler (KL) divergence, were a likely cause of the accuracy decline. We analyzed the data instability and our results showed that instability of spatial activation patterns and instability of the hemodynamic response in the most activated region are correlated with accuracy decline. Finally, we propose a method for mitigating classification instability in fNIRS-based emotion recognition based on feature selection for stable features, the first such method to our knowledge. This new feature selection criterion considers not only the separability of features (evaluated by Fisher Score) but also their stability over time (evaluated by KL divergence between feature distributions at different time points). Testing showed that this method led to an approximately 5% improvement in cross-session generalization accuracy.

Quantitative Estimation of Differentiated Mental Fatigue between Self-Rising Transfer and Multiple Welfare Robots-Assisted Rising Transfer

“…Therefore, in this paper, we investigated experiments between the two tasks with different difficulty levels. [31,32]. The signal mean, signal slope, signal variance, signal peak, signal kurtosis, and signal skewness were extracted as classification features.…”

“…To acquire brain signals resulting from mental arithmetic tasks, 16 detectors are positioned over the prefrontal cortex, in which each channel corresponds to a specific area of the prefrontal lobe. Multiple notch filters are applied to reduce physiological noises [31,32]. In the measurement process, the signal acquisition equipment is worn on the head, and the signal is transmitted to the PC through Wi-Fi.…”

“…∆c HbR (t) and ∆c HbO (t) signals were obtained by Equation (1). Based on the notch filter with band-reject ranges in Figure 3, 0.1-0.3 Hz signals were acquired and analyzed [31,32]. The signal mean, signal slope, signal variance, signal peak, signal kurtosis, and signal skewness were extracted as classification features.…”

Sensors and Systems for Physical Rehabilitation and Health Monitoring—A Review

“…However, emotion-related neural signals such as functional near-infrared spectroscopy (fNIRS) can be affected by various psychophysiological and environmental factors, resulting in data instability and classification instability, and this phenomenon can lead to dissatisfaction and abandonment of the system. In [119], the authors proposed a method to mitigate the instability of the classification in recognition of emotions based on fNIRS, using the selection of characteristics for stable characteristics. Preliminary tests showed that this method led to an improvement of approximately 5% in accuracy.…”

Functional near-infrared spectroscopy for intraoperative brain mapping

“…12 In recent years, fNIRS has advanced significantly and the common scalp probe has evolved into a relatively smaller and higher spatial resolution probe. 13 While fNIRS has been applied intraoperatively to study corticocortical activity in eloquent brain regions 14 and stereotactic functional localization, 11 its nonoperative applications continue to expand [15][16][17][18][19][20][21][22][23][24][25][26][27] and its value in surgical brain mapping remains limited. The present study explores the feasibility of fNIRS for intraoperative functional brain mapping using an enhanced probe and DCS for validation.…”