Effect of sensor fusion for recognition of emotional states using voice, face image and thermal image of face

Yoshitomi, Yasunari; Kim, Sung-Ill; Kawano, Takuya; Kilazoe, T.

doi:10.1109/roman.2000.892491

Cited by 98 publications

(71 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…More advanced systems are multi-modal and use a variety of microphones, video cameras as well as other sensors to enlighten the machine with richer signals from the human [10,11,12]. The above list of references is representative of existing work and not exhaustive.…”

Section: Introductionmentioning

confidence: 99%

Gesture-Based Affective Computing on Motion Capture Data

Kapur

Virji‐Babul

et al. 2005

Lecture Notes in Computer Science

164

View full text Add to dashboard Cite

Abstract. This paper presents research using full body skeletal movements captured using video-based sensor technology developed by Vicon Motion Systems, to train a machine to identify different human emotions. The Vicon system uses a series of 6 cameras to capture lightweight markers placed on various points of the body in 3D space, and digitizes movement into x, y, and z displacement data. Gestural data from five subjects was collected depicting four emotions: sadness, joy, anger, and fear. Experimental results with different machine learning techniques show that automatic classification of this data ranges from 84% to 92% depending on how it is calculated. In order to put these automatic classification results into perspective a user study on the human perception of the same data was conducted with average classification accuracy of 93%.

show abstract

Section: Introductionmentioning

confidence: 99%

Gesture-Based Affective Computing on Motion Capture Data

Kapur

Virji‐Babul

et al. 2005

Lecture Notes in Computer Science

164

View full text Add to dashboard Cite

show abstract

“…18 Consequently, the human face region is easily extracted from an image using the value of 1 for. [1][2][3][4][5][6][7][8][9][10][11][12] In principle, IR temperature measurements do not depend on skin color 19 , darkness, or lighting condition, and so the face region and its characteristics are easily extracted from a thermal image. Figure 1 shows a flowchart of the proposed method, which consists of two modules.…”

Section: Image Acquisitionmentioning

confidence: 99%

“…In order to avoid this problem and develop a method for facial expression recognition that is applicable under widely varying lighting conditions, we used images produced by infrared rays (IR), which reveal the thermal distribution of the face. [1][2][3][4][5][6][7][8][9][10][11][12] We adopted an utterance as the key to expressing human feelings or mental states because humans tend to express feelings vocally. [3][4][5][6][7][8][9][10][11][12] Although several studies on facial expression recognition using thermal image processing have been reported (See Refs.…”

Section: Introductionmentioning

confidence: 99%

“…[1][2][3][4][5][6][7][8][9][10][11][12] We adopted an utterance as the key to expressing human feelings or mental states because humans tend to express feelings vocally. [3][4][5][6][7][8][9][10][11][12] Although several studies on facial expression recognition using thermal image processing have been reported (See Refs. 1-17), only our research [3][4][5][6][7][8][9][10][11][12] has focused on the speaker.…”

Section: Introductionmentioning

confidence: 99%

“…[3][4][5][6][7][8][9][10][11][12] Although several studies on facial expression recognition using thermal image processing have been reported (See Refs. 1-17), only our research [3][4][5][6][7][8][9][10][11][12] has focused on the speaker. The present study also focuses on the speaker.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Facial Expression Recognition Using Thermal Image Processing and Efficient Preparation of Training-data

Nakanishi¹,

Yoshitomi²,

Asada³

et al. 2015

JRNAL

View full text Add to dashboard Cite

Using our previously developed system, we investigated the influence of training data on the facial expression accuracy using the training data of "taro" for the intentional facial expressions of "angry," "sad," and "surprised," and the training data of respective pronunciation for the intentional facial expressions of "happy" and "neutral." Using the proposed method, the facial expressions were discriminable with average accuracy of 72.4% for "taro," "koji" and "tsubasa", for the three facial expressions of "happy," "neutral," and "other".

show abstract

Significant Accomplishments, New Challenges, and New Perspectives

D’Mello

Calvo²

2011

New Perspectives on Affect and Learning Technologies

View full text Add to dashboard Cite

Effect of sensor fusion for recognition of emotional states using voice, face image and thermal image of face

Cited by 98 publications

References 3 publications

Gesture-Based Affective Computing on Motion Capture Data

Gesture-Based Affective Computing on Motion Capture Data

Facial Expression Recognition Using Thermal Image Processing and Efficient Preparation of Training-data

Significant Accomplishments, New Challenges, and New Perspectives

Contact Info

Product

Resources

About