Facial expressions analysis under occlusions based on specificities of facial motion propagation

Abstract

Although much progress has been made in the facial expression analysis field, facial occlusions are still challenging. The main innovation brought by this contribution consists in exploiting the specificities of facial movement propagation for recognizing expressions in presence of important occlusions. The movement induced by an expression extends beyond the movement epicenter. Thus, the movement occurring in an occluded region propagates towards neighboring visible regions. In presence of occlusions, per expression, we compute the importance of each unoccluded facial region and we construct adapted facial frameworks that boost the performance of per expression binary classifier. The output of each expression-dependant binary classifier is then aggregated and fed into a fusion process that aims constructing, per occlusion, a unique model that recognizes all the facial expressions considered. The evaluations highlight the robustness of this approach in presence of significant facial occlusions.

Dynamic Facial Expression Recognition under Partial Occlusion with Optical Flow Reconstruction

“…These different solutions are based on still images and do not exploit temporal aspects. Poux et al [15] have hypothesized that movement could bring more useful information and, especially, dense movement on which a property of propagation can be useful in the case of occlusion. To exploit this property, they propose to build a facial framework specific to the different partial occlusions from an initial facial framework of twenty-five regions.…”

“…Experimental protocol Dapogny et al [4] Out of bag error (OOB) Poux et al [15] 10 folds cross validation Li et al [12] 10 folds cross validation Cornejo et al [3] 80% for training, 20% for testing. 50% of training data is occluded, 50% of testing data is occluded Evaluations in the literature do not allow a fair and easy comparison.…”

“…The CK+ dataset has the advantage of being completely controlled in order to focus on the challenge of occlusion. Moreover this dataset is majoritarily used in the literature [3], [4], [12], [15] for exploring the challenge of occlusion and allows a comparison with other methods.…”

“…In order to evaluate the recognition step, we choose ten folds cross validation protocol which is mostly used in the literature [4], [12], [15]. The dataset is cut in ten stratified folds which contains proportional number of each facial expression compared with the original dataset.…”

Deep Learning for Micro-expression Recognition: A Survey

“…4) Optical flow: Optical flow is an approximation of the local image motion which has been verified helpful for motion representation [153]. The optical-flow based facial movement analysis [154], [155] offers a promising way for expression analysis with subtle movements [156], [157], various head poses [158] as well as facial occlusions [159]. Optical flow specifies the magnitude and direction of image pixel moves in a given sequence of images with a two-dimension vector field (horizontal and vertical optical flows).…”