A brain-computer interface for potential non-verbal facial communication based on EEG signals related to specific emotions

Abstract

Unlike assistive technology for verbal communication, the brain-machine or brain-computer interface (BMI/BCI) has not been established as a non-verbal communication tool for amyotrophic lateral sclerosis (ALS) patients. Face-to-face communication enables access to rich emotional information, but individuals suffering from neurological disorders, such as ALS and autism, may not express their emotions or communicate their negative feelings. Although emotions may be inferred by looking at facial expressions, emotional prediction for neutral faces necessitates advanced judgment. The process that underlies brain neuronal responses to neutral faces and causes emotional changes remains unknown. To address this problem, therefore, this study attempted to decode conditioned emotional reactions to neutral face stimuli. This direction was motivated by the assumption that if electroencephalogram (EEG) signals can be used to detect patients' emotional responses to specific inexpressive faces, the results could be incorporated into the design and development of BMI/BCI-based non-verbal communication tools. To these ends, this study investigated how a neutral face associated with a negative emotion modulates rapid central responses in face processing and then identified cortical activities. The conditioned neutral face-triggered event-related potentials that originated from the posterior temporal lobe statistically significantly changed during late face processing (600–700 ms) after stimulus, rather than in early face processing activities, such as P1 and N170 responses. Source localization revealed that the conditioned neutral faces increased activity in the right fusiform gyrus (FG). This study also developed an efficient method for detecting implicit negative emotional responses to specific faces by using EEG signals. A classification method based on a support vector machine enables the easy classification of neutral faces that trigger specific individual emotions. In accordance with this classification, a face on a computer morphs into a sad or displeased countenance. The proposed method could be incorporated as a part of non-verbal communication tools to enable emotional expression.

Emotional expression in psychiatric conditions: New technology for clinicians

“…We have tried to include the diagnostic and therapeutic methods that have been published in scientific journals, but the data, including quantitative and qualitative information, are limited, which causes an important drawback of this paper. For the summary of these methods see Table . What we can expect in the future is a rapid growth of use of modern technologies, such as VR, augmented reality, and interactive systems based on game consoles, in therapy enhancement.…”

Review and Classification of Emotion Recognition Based on EEG Brain-Computer Interface System Research: A Systematic Review

“…These proposed systems aim to explore or improve EEG-based emotion recognition systems. [2,39,41,42,49,50,57,61,63,92,104,108,109,117,131,136,149,152,157,173,174,185,186,189,191,[195][196][197][198][199][200][201][202][203][204][205][206][207][208][209]217,219,[223][224][225]229,[262][263][264][265][266]<...>…”

Integrating EEG and MEG Signals to Improve Motor Imagery Classification in Brain–Computer Interface

“…Brain-computer interfaces (BCIs) exploit the ability of subjects to modulate their brain activity through intentional mental effort, such as in motor imagery (MI). BCIs are increasingly used for control and communication, [1][2][3][4][5][6][7][8][9][10][11] and for the treatment of neurological disorders. [12][13][14][15][16][17] Despite their societal and clinical impact, many engineering challenges remain, from the optimization of the control features to the identification of the best mental strategy to code the user's intent.…”