No abstract

Abstract-This paper concerns the automatic recognition of human facial expressions using a fast 3D sensor, such as the Kinect. Facial expressions represent a rich source of information regarding emotion and interpersonal communication. The ability to recognize expressions automatically will have a large impact in many areas, particularly human-computer interaction. This paper describes 2 frameworks for recognizing 6 basic expressions using 3-dimensional data sequences that are captured in real time.Results are presented that demonstrate accuracy levels for the different techniques, and for different methods of preprocessing, registration and classification. We also describe the potential to use such a system for treatment of children with autism spectrum disorders (ASD).

Human facial expressions have been extensively studied using 2D static images or 2D video sequences. The main limitations of 2D-based analysis are problems associated with large variations in pose and illumination. Therefore, an alternative is to utilize depth information, captured from 3D sensors, which is both pose and illumination invariant. The Kinect sensor is an inexpensive, portable, and fast way to capture the depth information. However, only a few researchers have utilized the Kinect sensor for the automatic recognition of facial expressions. This is partly due to the lack of a Kinect-based publicly available RGBD facial expression recognition (FER) dataset that contains the relevant facial expressions and their associated semantic labels. This paper addresses this problem by presenting the first publicly available RGBD+time facial expression recognition dataset using the Kinect 1.0 sensor in both scripted (acted) and unscripted (spontaneous) scenarios. Our fully annotated dataset includes seven expressions (happiness, sadness, surprise, disgust, fear, anger, and neutral) for 32 subjects (males and females) aged from 10 to 30 and with different skin tones. Both human and machine evaluation were conducted. Each scripted expression was ranked quantitatively by two research assistants in the Psychology department. Baseline machine evaluation resulted in average recognition accuracy levels of 60% and 58.3% for 6 expressions and 7 expressions recognition, respectively, when features from 2D and 3D data were combined.978-1-4799-7824-3/15/$31.00

No abstract