Classification of Individual’s discrete emotions reflected in facial microexpressions using electroencephalogram and facial electromyogram

Kim, Hodam; Zhang, Dan; Kim, Laehyun; Im, Chang-Hwan

doi:10.1016/j.eswa.2021.116101

Cited by 20 publications

(8 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the EMG signals generated by activating specific forearm muscles can be conducted to the surroundings. [ 22 ] The EMG signals recorded by the two devices show similar patterns, as shown in Figure S7 (Supporting Information). Therefore, we confirm that these devices sat near the same muscle groups on the forearm.…”

Section: Resultsmentioning

confidence: 82%

See 1 more Smart Citation

AR‐Enabled Persistent Human–Machine Interfaces via a Scalable Soft Electrode Array

Kim,

Cha,

Kim

et al. 2023

Advanced Science

Self Cite

View full text Add to dashboard Cite

Augmented reality (AR) is a computer graphics technique that creates a seamless interface between the real and virtual worlds. AR usage rapidly spreads across diverse areas, such as healthcare, education, and entertainment. Despite its immense potential, AR interface controls rely on an external joystick, a smartphone, or a fixed camera system susceptible to lighting. Here, an AR‐integrated soft wearable electronic system that detects the gestures of a subject for more intuitive, accurate, and direct control of external systems is introduced. Specifically, a soft, all‐in‐one wearable device includes a scalable electrode array and integrated wireless system to measure electromyograms for real‐time continuous recognition of hand gestures. An advanced machine learning algorithm embedded in the system enables the classification of ten different classes with an accuracy of 96.08%. Compared to the conventional rigid wearables, the multi‐channel soft wearable system offers an enhanced signal‐to‐noise ratio and consistency over multiple uses due to skin conformality. The demonstration of the AR‐integrated soft wearable system for drone control captures the potential of the platform technology to offer numerous human–machine interface opportunities for users to interact remotely with external hardware and software.

show abstract

Section: Resultsmentioning

confidence: 82%

“…Then, the signals are further processed using a fourth‐order Butterworth bandpass filter with 20 and 450 Hz cutoff frequencies. [ 22,24 ] The filtered EMG signals are divided into short segments using a 300‐ms sliding window (600 samples) with 96% overlap. For each segment, a spatial covariance matrix (SCM) was extracted.…”

Section: Resultsmentioning

confidence: 99%

AR‐Enabled Persistent Human–Machine Interfaces via a Scalable Soft Electrode Array

Kim,

Cha,

Kim

et al. 2023

Advanced Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Most of the works using EMG for the recognition of emotional reactions focus on the analysis of facial expressions. For example, Kim et al [ 64 ] explored the use of facial EMG and EEG signals for the classification of the emotions of happiness, surprise, fear, anger, sadness, and disgust. Mithbavkar et al [ 65 ] developed a dataset for emotion recognition based on data collected through electromyograms using dance to stimulate emotional responses such as astonishment, awe, humor, and tranquility.…”

Section: Emotion Recognitionmentioning

confidence: 99%

Applying Self-Supervised Representation Learning for Emotion Recognition Using Physiological Signals

Quispe

Utyiama

Santos

et al. 2022

Sensors

View full text Add to dashboard Cite

The use of machine learning (ML) techniques in affective computing applications focuses on improving the user experience in emotion recognition. The collection of input data (e.g., physiological signals), together with expert annotations are part of the established standard supervised learning methodology used to train human emotion recognition models. However, these models generally require large amounts of labeled data, which is expensive and impractical in the healthcare context, in which data annotation requires even more expert knowledge. To address this problem, this paper explores the use of the self-supervised learning (SSL) paradigm in the development of emotion recognition methods. This approach makes it possible to learn representations directly from unlabeled signals and subsequently use them to classify affective states. This paper presents the key concepts of emotions and how SSL methods can be applied to recognize affective states. We experimentally analyze and compare self-supervised and fully supervised training of a convolutional neural network designed to recognize emotions. The experimental results using three emotion datasets demonstrate that self-supervised representations can learn widely useful features that improve data efficiency, are widely transferable, are competitive when compared to their fully supervised counterparts, and do not require the data to be labeled for learning.

show abstract

“…Extracting features from fEMG data entails identifying distinct elements within the signal that reflect different emotional states. Several conventional methods for feature extraction in fEMG analysis encompass time-domain [5][6][7][8][9][10], frequency-domain [11], and time-frequency domain [12,13]. Computing time, frequency, and time-frequency features can provide valuable insights and information about the characteristics of signals, data, and phenomena, allowing for better understanding, analysis, and decision-making [14].…”

Section: Introductionmentioning

confidence: 99%

Evaluating the Effectiveness of Machine Learning in Identifying the Optimal Facial Electromyography Location for Emotion Detection

Kumar,

PJ,

et al. 2023

Preprint

View full text Add to dashboard Cite

Introduction: Emotional state recognition is crucial for identifying emotions and providing valuable insights into detecting prolonged stress or negative emotions in individuals. In this study, we explore the feasibility of utilizing facial electromyography (fEMG) signals to accurately recognize the emotions and determine the optimal recording location. Materials and methods: To investigate various emotions, we used the continuously annotated signals of emotion dataset, consisting of fEMG signals captured from three distinct muscle locations: zygomaticus major (zEMG), corrugator supercilii (cEMG), and trapezius (tEMG). These fEMG signals underwent analysis through feature extraction in the time, frequency, and time-frequency domains. We identified the optimal muscle location for recognizing emotions using different machine learning models, such as logistic regression (LR), support vector machine, and random forest (RF), and validated the results using a 10-fold cross-validation approach. Additionally, we identified the most influential features for distinguishing between the emotions using the RF feature ranking method. Results: Our findings showed that we attained the highest average accuracy of 74.79% for emotion classification by utilizing 31 top-ranked features from the time, frequency, and time-frequency domains of three fEMG signals (zEMG, cEMG, and tEMG) with the RF classifier. Moreover, we achieved an average accuracy of 74.17% by utilizing the top 10 time-domain features extracted from the zEMG signals with the LR classifier. In summary, this study demonstrates promising outcomes in utilizing fEMG signals for efficient emotion recognition and presents an innovative approach to the field of affective computing.

show abstract

Classification of Individual’s discrete emotions reflected in facial microexpressions using electroencephalogram and facial electromyogram

Cited by 20 publications

References 23 publications

AR‐Enabled Persistent Human–Machine Interfaces via a Scalable Soft Electrode Array

AR‐Enabled Persistent Human–Machine Interfaces via a Scalable Soft Electrode Array

Applying Self-Supervised Representation Learning for Emotion Recognition Using Physiological Signals

Evaluating the Effectiveness of Machine Learning in Identifying the Optimal Facial Electromyography Location for Emotion Detection

Contact Info

Product

Resources

About