A Novel Supervised Bimodal Emotion Recognition Approach Based on Facial Expression and Body Gesture

Yan, Jun; Lu, Guanming; Bai, Xiaodong; Li, Haibo; Sun, Ning; Liang, Ruiyu

doi:10.1587/transfun.e101.a.2003

Cited by 8 publications

(4 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They offers a compelling platform for touchless interactions. The advantages and applications of face detection are well-explored in the literature [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 ]. The advantages of facial expressions including the Face Switch have been well documented and investigated in research [ 9 ].…”

Section: Introductionmentioning

confidence: 99%

Hybrid Target Selections by ”Hand Gestures + Facial Expression” for a Rehabilitation Robot

Han

Zhang

et al. 2022

Sensors

View full text Add to dashboard Cite

In this study we propose a “hand gesture + face expression” human machine interaction technique, and apply this technique to bedridden rehabilitation robot. “Hand gesture + Facial expression” interactive technology combines the input mode of gesture and facial expression perception. It involves seven basic facial expressions that can be used to determine a target selecting task, while hand gestures are used to control a cursor’s location. A controlled experiment was designed and conducted to evaluate the effectiveness of the proposed hybrid technology. A series of target selecting tasks with different target widths and layouts were designed to examine the recognition accuracy of hybrid control gestures. An interactive experiment applied to a rehabilitation robot is designed to verify the feasibility of this interactive technology applied to rehabilitation robots. The experimental results show that the “hand + facial expression” interactive gesture has strong robustness, which can provide a novel guideline for designing applications in VR interfaces, and it can be applied to the rehabilitation robots.

show abstract

Section: Introductionmentioning

confidence: 99%

Hybrid Target Selections by ”Hand Gestures + Facial Expression” for a Rehabilitation Robot

Han

Zhang

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Affective computing is that the computer can automatically identify, understand and reflect human emotions. For example, computers can recognize emotions from a person's facial expressions [2], voice [3], [4], blinking [5] and posture [6], etc. Most previous studies have focused on voice and facial expressions.…”

Section: Introductionmentioning

confidence: 99%

Multimodal Emotion Recognition Based on Ensemble Convolutional Neural Network

Huang

Wen-ming

et al. 2020

IEEE Access

View full text Add to dashboard Cite

In recent years, emotional recognition based on Electrophysiological (EEG) signals has become more and more popular. But the researchers ignored the fact that peripheral physiological signals can also reflect changes in mood. We propose an Ensemble Convolutional Neural Network (ECNN) model, which is used to automatically mine the correlation between multi-channel EEG signals and peripheral physiological signals in order to improve the emotion recognition accuracy. First, we design five convolution networks and use global average pooling (GAP) layers instead of fully connected layers; and then the plurality voting strategy is adopted to establish the ensemble model; eventually this model divides emotions into four categories. Based on the simulations on DEAP dataset, the experimental results demonstrate the superiority of the ECNN compared with other methods.

show abstract

“…In the research domain of affective computing and computer vision, the research of facial expression recognition, speech emotion recognition and physiological signals emotion recognition which are only based on one modality is more and more replaced by the bimodal emotion recognition and multimodal emotion recognition which are based on two modalities, three modalities or even more modalities in recent years [1]- [8]. No matter for facial expression recognition, speech emotion recognition, physiological signals emotion recognition or bimodal emotion recognition and multimodal emotion recognition, the establishment of the corresponding emotion database is the most primary and important step [9]. We can not conduct the emotion recognition research smoothly and deeply in case of lacking the dependable emotion database [10]- [14].…”

Section: Introductionmentioning

confidence: 99%

“…In the wake of the increase of the emotion database based on one modality, a few researchers attempt to focus on the establishment of bimodal emotion database and multimodal emotion database [9], [26]- [29]. Compared to the emotion database based on one modality, establishing a dependable bimodal emotion database and multimodal emotion database is more difficult and complicated, because the channel of the emotion data recording is increased to two, three or even more, and the data recording of each channel should be coinstantaneous [9], [26], [27]. As so far, a few effective bimodal emotion databases and multimodal emotion databases such as FABO and DEAP have been established on the basis of those emotion databases using one modality in last several years [9], [26]- [34].…”

Section: Introductionmentioning

confidence: 99%

A Novel Bimodal Emotion Database from Physiological Signals and Facial Expression

Yan

Wang

Liang

2018

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

In this paper, we establish a novel bimodal emotion database from physiological signals and facial expression, which is named as PSFE. The physiological signals and facial expression of the PSFE database are respectively recorded by the equipment of the BIOPAC MP 150 and the Kinect for Windows in the meantime. The PSFE database altogether records 32 subjects which include 11 women and 21 man, and their age distribution is from 20 to 25. Moreover, the PSFE database records three basic emotion classes containing calmness, happiness and sadness, which respectively correspond to the neutral, positive and negative emotion state. The general sample number of the PSFE database is 288 and each emotion class contains 96 samples.

show abstract

A Novel Supervised Bimodal Emotion Recognition Approach Based on Facial Expression and Body Gesture

Cited by 8 publications

References 32 publications

Hybrid Target Selections by ”Hand Gestures + Facial Expression” for a Rehabilitation Robot

Hybrid Target Selections by ”Hand Gestures + Facial Expression” for a Rehabilitation Robot

Multimodal Emotion Recognition Based on Ensemble Convolutional Neural Network

A Novel Bimodal Emotion Database from Physiological Signals and Facial Expression

Contact Info

Product

Resources

About