Human expression recognition from motion using a radial basis function network architecture

Rosenblum, Mark L.; Yacoob, Yaser; Davis, Larry S.

doi:10.1109/72.536309

Cited by 234 publications

(107 citation statements)

References 16 publications

Supporting

Mentioning

106

Contrasting

Order By: Relevance

“…Facial expressions [20], [21], [22], vocal features [23] [24] [25], body movements and postures [26], [27], [11], [28], physiological signals [29] have been used as inputs during these attempts, although multimodal emotion recognition is currently gaining ground [7], [30], [31], [32], [33]. Nevertheless, most of the work has considered the integration of information from facial expressions and speech [34], [35] and there have been relatively few attempts to combine information from body movement and gestures in a multimodal framework.…”

Section: Related Workmentioning

confidence: 99%

Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis

Kessous¹,

Castellano

2009

J Multimodal User Interfaces

215

View full text Add to dashboard Cite

In this paper a study on multimodal automatic emotion recognition during a speech-based interaction is presented. A database was constructed consisting of people pronouncing a sentence in a scenario where they interacted with an agent using speech. Ten people pronounced a sentence corresponding to a command while making 8 different emotional expressions. Gender was equally represented, with speakers of several different native languages including French, German, Greek and Italian. Facial expression, gesture and acoustic analysis of speech were used to extract features relevant to emotion. For the automatic classification of unimodal data, bimodal data and multimodal data, a system based on a Bayesian classifier was used. After performing an automatic classification of each modality, the different modalities were combined using a multimodal approach. Fusion of the modalities at the feature level (before running the classifier) and at the results level (combining results from classifier from each modality) were compared. Fusing the multimodal data resulted in a large increase in the recognition rates in comparison to the unimodal systems: the multimodal approach increased the recognition rate by more than 10% when compared to the most successful unimodal system. Bimodal emotion recognition based on all combinations of the modalities (i.e., 'face-gesture', 'facespeech' and 'gesture-speech') was also investigated. The results show that the best pairing is 'gesture-speech'. Using all three modalities resulted in a 3.3% classification improvement over the best bimodal results.

show abstract

Section: Related Workmentioning

confidence: 99%

Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis

Kessous¹,

Castellano

2009

J Multimodal User Interfaces

215

View full text Add to dashboard Cite

show abstract

“…The midlevel representation was then classified into one of six facial expressions using a set of heuristic rules. Rosenblum et al~1996! expanded this work to analyze facial expressions using the full temporal profile of the expression, from initiation to apex and then to relaxation.…”

Section: Facial Action Codes Versus Emotion Categoriesmentioning

confidence: 99%

“…2 Recent advances have been made in computer vision for automatic recognition of facial expressions in images. The approaches that have been explored include analysis of facial motion~Essa & Pentland, 1997;Mase, 1991;Rosenblum, Yacoob, & Davis, 1996;Yacoob & Davis, 1994!, measurements of the shapes of facial features and their spatial arrangements~Lanitis, Taylor, & Cootes, 1997!, holistic spatial pattern analysis using techniques based on principal components analysis~PCA!~Cottrell & Metcalfe, 1991; Lanitis et al, 1997;Padgett & Cottrell, 1997!, and methods for relating face images to physical models of the facial skin and musculature~Essa & Pentland, 1997;Li, Roivainen, & Forcheimer, 1993;Mase, 1991;Terzopoulos & Waters, 1993!. These systems demonstrate approaches to face image analysis that are applicable to the present goals, but the systems themselves are of limited use for behavioral and psychophysiological research.…”

Section: Analysis Of Facial Signals By Computermentioning

confidence: 99%

Measuring facial expressions by computer image analysis

Hager²,

et al. 1999

View full text Add to dashboard Cite

Facial expressions provide an important behavioral measure for the study of emotion, cognitive processes, and social interaction. The Facial Action Coding System~Ekman & Friesen, 1978! is an objective method for quantifying facial movement in terms of component actions. We applied computer image analysis to the problem of automatically detecting facial actions in sequences of images. Three approaches were compared: holistic spatial analysis, explicit measurement of features such as wrinkles, and estimation of motion flow fields. The three methods were combined in a hybrid system that classified six upper facial actions with 91% accuracy. The hybrid system outperformed human nonexperts on this task and performed as well as highly trained experts. An automated system would make facial expression measurement more widely accessible as a research tool in behavioral science and investigations of the neural substrates of emotion.

show abstract

“…For these categories a number of acoustic features such as linear predictive coe cient, mel-frequency cepstral coe cients are extracted to characterize the audio content. Rosenblum et al [176] have developed an RBF architecture that learns the correlation of facial feature motion patterns and human expressions. Each expression network was trained by viewing a set of sequences of one expression for many subjects.…”

Section: Rbfns In Classi Cation and Predictionmentioning

confidence: 99%

Radial basis function neural networks: a topical state-of-the-art survey

Dash¹,

Behera²,

Dehuri

et al. 2016

Open Computer Science

View full text Add to dashboard Cite

Radial basis function networks (RBFNs) have gained widespread appeal amongst researchers and have shown good performance in a variety of application domains. They have potential for hybridization and demonstrate some interesting emergent behaviors. This paper aims to o er a compendious and sensible survey on RBF networks. The advantages they o er, such as fast training and global approximation capability with local responses, are attracting many researchers to use them in diversi ed elds. The overall algorithmic development of RBF networks by giving special focus on their learning methods, novel kernels, and ne tuning of kernel parameters have been discussed. In addition, we have considered the recent research work on optimization of multi-criterions in RBF networks and a range of indicative application areas along with some open source RBFN tools.

show abstract

Human expression recognition from motion using a radial basis function network architecture

Cited by 234 publications

References 16 publications

Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis

Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis

Measuring facial expressions by computer image analysis

Radial basis function neural networks: a topical state-of-the-art survey

Contact Info

Product

Resources

About