Photogram Classification-Based Emotion Recognition

Abstract

This paper presents a method for facial emotion recognition based on parameterized photograms and machine learning techniques. Videos of people displaying emotions are parameterized by means of a facial feature-based emotional category association process to determine whether a given photogram expresses emotions by comparing the facial action units displayed with findings in the literature about facial emotion. To test the proposed approach, two strategies are adopted. First, photograms displaying emotions are gathered, and then different machine learning classifiers are applied to check the goodness of the obtained set of categorized emotional photograms. Second, classifiers trained on the sets of emotional photograms were then used to emotionally classify all the videos in each database, using all the photograms with no preprocessing or photogram selection. The presented method was tested using the OpenFace parameterizer with emotional videos gathered from Multimedia Understanding Facial Expression (MUG) and Cohn-Kanade (CK+) databases. The outcomes achieved for emotional photogram classification on the sets of emotional photograms reached maximums of 99.80% and 99.63% in the MUG and CK+ databases, respectively. The videos were classified using different voting strategies regarding the outcome of each photogram in the video with all the photogram emotion recognition classifiers obtained results reflecting recognition rates of 70.71% and 66.36% for the videos in MUG and CK+ databases, and reached up to 72.55% and 88.37% when classifier permutation strategies were used. The work carried out opens the door to followup work concerning data preprocessing and the use of different classifier combination methods in facial emotion recognition.

Video Analysis Based Effective Multi-Facial Emotion Recognition and Classification Framework Using Gcrcnn

“…López-Gil et al [20] introduced the FER method using parameterized photograms and ML techniques. This model used facial feature-based emotional classification techniques and various classifiers to achieve high emotion recognition rates.…”

AI-Based Publicity Strategies for Medical Colleges: A Case Study of Healthcare Analysis

“…In (31), a new two-layer feature selection framework was proposed for emotion classification from a comprehensive list of body motion features. In (32), a face emotion recognition method based on parametric images and machine learning technology was proposed. In (33), a new group emotion recognition method was proposed to estimate the group emotion.…”