PPG and EMG Based Emotion Recognition using Convolutional Neural Network

Lee, Min Seop; Cho, Ye Ri; Lee, Yun Kyu; Pae, Dong Sung; Lim, Myo Taeg; Kang, Tae-Koo

doi:10.5220/0007797005950600

Cited by 9 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Udovičić et al [ 20 ] studied emotion recognition using only GSR and PPG signals because of their suitability for implementation in a simple wearable device that can collect signals from a person without compromising comfort and privacy. In addition to the above studies, there have been many others on the use of PPG data for emotion recognition [ 21 , 22 , 23 ].…”

Section: Related Workmentioning

confidence: 99%

PPG2EMG: Estimating Upper-Arm Muscle Activities and EMG from Wrist PPG Values

Okamoto

Murao

2023

Sensors

View full text Add to dashboard Cite

The electromyogram (EMG) is a waveform representation of the action potential generated by muscle cells using electrodes. EMG acquired using surface electrodes is called surface EMG (sEMG), and it is the acquisition of muscle action potentials transmitted by volume conduction from the skin. Surface electrodes require disposable conductive gel or adhesive tape to be attached to the skin, which is costly to run, and the tape is hard on the skin when it is removed. Muscle activity can be evaluated by acquiring muscle potentials and analyzing quantitative, temporal, and frequency factors. It is also possible to evaluate muscle fatigue because the frequency of the EMG becomes lower as the muscle becomes fatigued. Research on human activity recognition from EMG signals has been actively conducted and applied to systems that support arm and hand functions. This paper proposes a method for recognizing the muscle activity state of the arm using pulse wave data (PPG: Photoplethysmography) and a method for estimating EMG using pulse wave data. This paper assumes that the PPG sensor is worn on the user’s wrist to measure the heart rate. The user also attaches an elastic band to the upper arm, and when the user exerts a force on the arm, the muscles of the upper arm contract. The arteries are then constricted, and the pulse wave measured at the wrist becomes weak. From the change in the pulse wave, the muscle activity of the arm can be recognized and the number of action potentials of the muscle can be estimated. From the evaluation experiment with five subjects, three types of muscle activity were recognized with 80+%, and EMG was estimated with approximately 20% error rate.

show abstract

Section: Related Workmentioning

confidence: 99%

PPG2EMG: Estimating Upper-Arm Muscle Activities and EMG from Wrist PPG Values

Okamoto

Murao

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…The target emotions of this study were joy, sadness, anger, and relaxed, such that the emotions represent each quadrant of the emotional space of arousal and valence (see Figure 1). This emotional space has been widely utilized in emotional recognition studies [22,[28][29][30]32,33]. The target emotions of this study were joy, sadness, anger, and relaxed, such that the emotions represent each quadrant of the emotional space of arousal and valence (see Figure 1).…”

Section: Dataset Acquisitionmentioning

confidence: 99%

“…The target emotions of this study were joy, sadness, anger, and relaxed, such that the emotions represent each quadrant of the emotional space of arousal and valence (see Figure 1). This emotional space has been widely utilized in emotional recognition studies [22,[28][29][30]32,33]. The data were recorded from a pulse sensor [26] connected to Arduino with the sampling rate set to 100 Hz, while participants watched a video clip.…”

Section: Dataset Acquisitionmentioning

confidence: 99%

“…The sampling rate of 100 Hz is relatively lower than the other dataset, but it still covers the most frequency bands of interests. Features have been extracted in 0.67-50 Hz band for ECG in general [12], and recent studies utilized the frequency lower than 0.4 Hz for emotion recognition with PPG [29,33,34]. Figure 2 shows a sample PPG signal of the obtained data.…”

Section: Dataset Acquisitionmentioning

confidence: 99%

See 1 more Smart Citation

A Photoplethysmogram Dataset for Emotional Analysis

et al. 2022

View full text Add to dashboard Cite

In recent years, research on emotion classification based on physiological signals has actively attracted scholars’ attention worldwide. Several studies and experiments have been conducted to analyze human emotions based on physiological signals, including the use of electrocardiograms (ECGs), electroencephalograms (EEGs), and photoplethysmograms (PPGs). Although the achievements with ECGs and EEGs are progressive, reaching higher accuracies over 90%, the number of studies utilizing PPGs are limited and their accuracies are relatively lower than other signals. One of the difficulties in studying PPGs for emotional analysis is the lack of open datasets (there is a single dataset to the best of the authors). This study introduces a new PPG dataset for emotional analysis. A total of 72 PPGs were recorded from 18 participants while watching short video clips and analyzed in time and frequency domains. Moreover, emotional classification accuracies with the presented dataset were presented with various neural network structures. The results prove that this dataset can be used for further emotional analysis with PPGs.

show abstract

“…Human–robot interaction involves understanding how people behave and feel, as this helps the robot to interact with people in an appropriate manner [ 3 ]. Human emotion techniques have been used in security applications to identify people who are able to mask their emotions, often referred to as having a “poker face” [ 4 ], and in deception detection to detect when someone has not been truthful or accurate in their statements [ 5 , 6 ]. They have also been used for medical applications, including people with autism disorder who may not be able to use their body language, facial expressions, or spoken language to show how they feel.…”

Section: Introductionmentioning

confidence: 99%

Analysis of Facial Occlusion Challenge in Thermal Images for Human Affective State Recognition

Qudah

Mohamed

Lutfi

2023

Sensors

View full text Add to dashboard Cite

Several studies have been conducted using both visual and thermal facial images to identify human affective states. Despite the advantages of thermal facial images in recognizing spontaneous human affects, few studies have focused on facial occlusion challenges in thermal images, particularly eyeglasses and facial hair occlusion. As a result, three classification models are proposed in this paper to address the problem of thermal occlusion in facial images, with six basic spontaneous emotions being classified. The first proposed model in this paper is based on six main facial regions, including the forehead, tip of the nose, cheeks, mouth, and chin. The second model deconstructs the six main facial regions into multiple subregions to investigate the efficacy of subregions in recognizing the human affective state. The third proposed model in this paper uses selected facial subregions, free of eyeglasses and facial hair (beard, mustaches). Nine statistical features on apex and onset thermal images are implemented. Furthermore, four feature selection techniques with two classification algorithms are proposed for a further investigation. According to the comparative analysis presented in this paper, the results obtained from the three proposed modalities were promising and comparable to those of other studies.

show abstract

PPG and EMG Based Emotion Recognition using Convolutional Neural Network

Cited by 9 publications

References 0 publications

PPG2EMG: Estimating Upper-Arm Muscle Activities and EMG from Wrist PPG Values

PPG2EMG: Estimating Upper-Arm Muscle Activities and EMG from Wrist PPG Values

A Photoplethysmogram Dataset for Emotional Analysis

Analysis of Facial Occlusion Challenge in Thermal Images for Human Affective State Recognition

Contact Info

Product

Resources

About