Auto-Optimized Multimodal Expression Recognition Framework Using 3D Kinect Data for ASD Therapeutic Aid

Youssef, Amira E.; Aly, Sherin; Ibrahim, Ahmed S.; Abbott, A. Lynn

doi:10.7763/ijmo.2013.v3.247

Cited by 24 publications

(11 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The system proposed by (Youssef, Aly, Ibrahim, & Abbott, 2013) attempts to recognize the six basic emotions using a Microsoft Kinect sensor. At first, the system extracts 4D facial points (dynamic 3D facial points).…”

Section: Affect Sensing -Systems and Devicesmentioning

confidence: 99%

Mirroring and recognizing emotions through facial expressions for a RoboKind platform

Silva

Soares

Esteves

2017

2017 IEEE 5th Portuguese Meeting on Bioengineering (ENBENG)

View full text Add to dashboard Cite

Facial expressions play an important role during human social interaction, enabling communicative cues, ascertaining the level of interest or signalling the desire to take a speaking turn. They also give continuous feedback indicating that the information conveyed has been understood. However, certain individuals have difficulties in social interaction in particular verbal and non-verbal communication (e.g. emotions and gestures). Autism Spectrum Disorders (ASD) are a special case of social impairments. Individuals that are affected with ASD are characterized by repetitive patterns of behaviour, restricted activities or interests, and impairments in social communication. The use of robots had already been proven to encourage the promotion of social interaction and skills in children with ASD. Following this trend, in this work a robotic platform is used as a mediator in the social interaction activities with children with special needs. The main purpose of this dissertation is to develop a system capable of automatic detecting emotions through facial expressions and interfacing it with a robotic platform in order to allow social interaction with children with special needs. The proposed experimental setup uses the Intel RealSense 3D camera and the Zeno R50 Robokind robotic platform. This layout has two subsystems, a Mirroring Emotion System (MES) and an Emotion Recognition System (ERS). The first subsystem (MES) is capable of synthetizing human emotions through facial expressions, on-line. The other subsystem (ERS) is able to recognize human emotions through facial features in real time. MES extracts the user facial Action Units (AUs), sends the data to the robot allowing on-line imitation. ERS uses Support Vector Machine (SVM) technique to automatic classify the emotion expressed by the User in real time. Finally, the proposed subsystems, MES and ERS, were evaluated in a laboratorial and controlled environment in order to check the integration and operation of the systems. Then, both subsystems were tested in a school environment in different configurations. The results of these preliminary tests allowed to detect some constraints of the system, as well as validate its adequacy in an intervention setting.

show abstract

Section: Affect Sensing -Systems and Devicesmentioning

confidence: 99%

Mirroring and recognizing emotions through facial expressions for a RoboKind platform

Silva

Soares

Esteves

2017

2017 IEEE 5th Portuguese Meeting on Bioengineering (ENBENG)

View full text Add to dashboard Cite

show abstract

“…A small number of studies have used the Kinect depth sensor for face applications. These include the recognition of basic facial expressions [14] and the classification of different facial exercises [15]. In U.K., ongoing research uses the Kinect depth sensor for facial rehabilitation of patients following stroke [16].…”

Section: Introductionmentioning

confidence: 99%

A pilot study on automated quantitative grading of facial functions

Gaber¹,

Taher²,

Wahed³

2020

Vib. proced.

View full text Add to dashboard Cite

Quantitative grading of facial paralysis (FP) and the associated loss of facial function are essential to evaluate the severity and to track deterioration or improvement of the condition following treatment. To date, several computer-assisted grading systems have been proposed but none have gained widespread clinical acceptance. There is still a need for an accurate quantitative assessment tool that is automatic, inexpensive, easy to use, and has low inter-observer variability. The aim of the authors is to develop such a comprehensive Automated Facial Grading (AFG) system. One of this system's modules: the resting symmetry module has already been presented. The present study describes the implementation of the second module for grading voluntary movements. The system utilizes the Kinect v2 sensor to detect and capture facial landmarks in real time. The functions of three regions, the eyebrows, eyes and mouth, are evaluated by quantitatively grading four voluntary movements. Preliminary results on normal subjects and patients are promising. The AFG system is a novel system that is suitable for clinical use because it is fast, objective, easy to use, and inexpensive. With further enhancement, it can be extended to become a virtual facial rehabilitation tool.

show abstract

“…Using these two features respectively, the classification of emotions has been done using support vector machine (SVM) classifiers and the recognition results of 30 consecutive frames are fused by the fusion algorithm based on improved emotional profiles (IEPs). In 2013, Youssef et al [10] constructed a dataset containing 3D data for 14 different persons performing the 6 basic facial expressions. SVM and k-NN are used to classify emotions.…”

Section: Introductionmentioning

confidence: 99%