The Effect of the Number of Key-Frames on the Facial Emotion Recognition Accuracy

Abstract

Key-frame selection plays an important role in facial expression recognition systems. It helps in selecting the most representative frames that capture the different poses of the face. The effect of the number of selected keyframes has been studied in this paper to find its impact on the final accuracy of the emotion recognition system. Dynamic and static information is employed to select the most effective key-frames of the facial video with a short response time. Firstly, the absolute difference between the successive frames is used to reduce the number of frames and select the candidate ones which then contribute to the clustering process. The static-based information of the reduced sets of frames is then given to the fuzzy C-Means algorithm to select the best C-frames. The selected keyframes are then fed to a graph mining-based facial emotion recognition system to select the most effective sub-graphs in the given set of keyframes. Different experiments have been conducted using Surrey Audio-Visual Expressed Emotion (SAVEE) database and the results show that the proposed method can effectively capture the keyframes that give the best accuracy with a mean response time equals to 2.89s.

Internet and Biometric Web Based Business Management Decision Support

“… Among regression algorithms the usual choices are: linear regression [173,174,175], Lasso Regression [176,177], Logistic Regression [178,179,180], Multivariate Regression [181,182], and Multiple Regression Algorithm [183,184].  Among clustering algorithms the most common choices in biometrics or neuroscience research are: K-Means Clustering [185,186,187], Fuzzy C-means Algorithm [188,189], Expectation-Maximization (EM) Algorithm [190], and Hierarchical Clustering Algorithm [188,191,192].  Among reinforcement learning algorithms the most common choices are: deep reinforcement learning [193,194,195] and inverse reinforcement learning [196].…”

Localization of Strangeness for Real Time Video in Crowd Activity Using Optical Flow and Entropy

“…Keep in vision that [22], if camera position parameters are given, the optical flow module may also be utilized for extracting parallax in static stereoscopic vision [23]. Based on the presumptions that pixel intensities don't fluctuate over time and that nearby pixels move similarly, optical flow operates [24]. Additional presumptions can include the motion being locally smooth or the visual gradients being constant.…”

A Review of AI Cloud and Edge Sensors, Methods, and Applications for the Recognition of Emotional, Affective and Physiological States

“… Among regression algorithms the usual choices are: linear regression [ 173 , 174 , 175 ], Lasso Regression [ 176 , 177 ], Logistic Regression [ 178 , 179 , 180 ], Multivariate Regression [ 181 , 182 ], and Multiple Regression Algorithm [ 183 , 184 ]. Among clustering algorithms the most common choices in biometrics or neuroscience research are: K-Means Clustering [ 185 , 186 , 187 ], Fuzzy C-means Algorithm [ 188 , 189 ], Expectation-Maximization (EM) Algorithm [ 190 ], and Hierarchical Clustering Algorithm [ 188 , 191 , 192 ]. Among reinforcement learning algorithms the most common choices are: deep reinforcement learning [ 193 , 194 , 195 ] and inverse reinforcement learning [ 196 ].…”

“…Among clustering algorithms the most common choices in biometrics or neuroscience research are: K-Means Clustering [ 185 , 186 , 187 ], Fuzzy C-means Algorithm [ 188 , 189 ], Expectation-Maximization (EM) Algorithm [ 190 ], and Hierarchical Clustering Algorithm [ 188 , 191 , 192 ].…”