An Evolutive Approach for Smile Recognition in Video Sequences

Abstract

Facial expression recognition is one of the most challenging research areas in the image recognition field and has been actively studied since the 70's. For instance, smile recognition has been studied due to the fact that it is considered an important facial expression in human communication, it is therefore likely useful for human-machine interaction. Moreover, if a smile can be detected and also its intensity estimated, it will raise the possibility of new applications in the future. We are talking about quantifying the emotion at low computation cost and high accuracy. For this aim, we have used a new support vector machine (SVM) based approach that integrates a weighted combination of local binary patterns (LBPs) and principal component analysis (PCA) based approaches. Furthermore, we construct this smile detector considering the evolution of the emotion along its natural life cycle. As a consequence, we achieved both low computation cost and high performance with video sequences.

Facial expression analysis in a wild sporting environment

“…In static-based methods, the feature representation is only encoded with spatial information from the current single image [23,26]. In contrast, dynamics-based methods also consider the temporal relation between contiguous frames in the input sequence [12,19].…”

Automatic tool to annotate smile intensities in conversational face-to-face interactions

“…The proposed solutions vary indeed whether one wants to detect the presence or the absence of smile (An, Yang, & Bhanu, 2015;Chen, Ou, Chi, & Fu, 2017;Guo, Polania, & Barner, 2018;Shan, 2012;Zhang, Huang, Wu, & Wang, 2015) or rather one wants to estimate smile intensity (Bartlett, Littlewort, Braathen, Sejnowski, & Movellan, 2003;Bartlett et al, 2006;Girard, Cohn, & De la Torre, 2015;Jiang, Coskun, Badokhon, Liu, & Huang, 2019;Shimada, Matsukawa, Noguchi, & Kurita, 2010;Vinola & Vimala Devi, 2019). The methods applied also change if one is interested in classifying single face image (An et al, 2015;Chen et al, 2017;Guo et al, 2018;Jiang et al, 2019;Shan, 2012;Shimada et al, 2010;Zhang et al, 2015) rather than proposing a dynamical annotation of a video recording (Freire-Obregón & Castrillón-Santana, 2015). The main difficulty plaguing in practice the automatic smile intensity estimation task lies however on the lack of a large dataset with manually annotated references (Girard et al, 2015;Guo et al, 2018;Walecki, Rudovic, Pavlovic, & Pantic, 2019).…”