Uncertainty Flow Facilitates Zero-Shot Multi-Label Learning in Affective Facial Analysis

Abstract

Featured Application: The proposed Uncertainty Flow framework may benefit the facial analysis with its promised elevation in discriminability in multi-label affective classification tasks. Moreover, this framework also allows the efficient model training and between tasks knowledge transfer. The applications that rely heavily on continuous prediction on emotional valance, e.g., to monitor prisoners' emotional stability in jail, can be directly benefited from our framework. Abstract:To lower the single-label dependency on affective facial analysis, it urges the fruition of multi-label affective learning. The impediment to practical implementation of existing multi-label algorithms pertains to scarcity of scalable multi-label training datasets. To resolve this, an inductive transfer learning based framework, i.e., Uncertainty Flow, is put forward in this research to allow knowledge transfer from a single labelled emotion recognition task to a multi-label affective recognition task. I.e., the model uncertainty-which can be quantified in Uncertainty Flow-is distilled from a single-label learning task. The distilled model uncertainty ensures the later efficient zero-shot multi-label affective learning. On the theoretical perspective, within our proposed Uncertainty Flow framework, the feasibility of applying weakly informative priors, e.g., uniform and Cauchy prior, is fully explored in this research. More importantly, based on the derived weight uncertainty, three sets of prediction related uncertainty indexes, i.e., soft-max uncertainty, pure uncertainty and uncertainty plus are proposed to produce reliable and accurate multi-label predictions. Validated on our manual annotated evaluation dataset, i.e., the multi-label annotated FER2013, our proposed Uncertainty Flow in multi-label facial expression analysis exhibited superiority to conventional multi-label learning algorithms and multi-label compatible neural networks. The success of our proposed Uncertainty Flow provides a glimpse of future in continuous, uncertain, and multi-label affective computing.

New Trends in Emotion Recognition Using Image Analysis by Neural Networks, a Systematic Review

“…The advantages of choosing a CNN for FER systems include its extremely high level of performance, the elimination of the manual feature extraction requirement since the learning is automatically performed on the training data, and perhaps the most important advantage, which is transfer learning, because CNNs allow subsequent constructions based on initial parts of other pre-trained CNNs [ 34 , 71 , 105 , 106 , 107 , 108 , 109 , 110 ]. Transfer learning can be extremely useful because information learned for one task can be transferred to another task, greatly reducing the processing time by eliminating the need to recollect training data for that given task.…”

Special Issue on Socio-Cognitive and Affective Computing

“…Results show the promising state of eFES as compared to the traditional feature selection process. Lastly, an inductive transfer learning-based framework (Uncertainty Flow) is put forward to allow knowledge transfer from a single-labeled emotion recognition task to a multi-label affective recognition task to lower the single-label dependency on affective facial analysis [18]. The authors demonstrate that Uncertainty Flow in multi-label facial expression analysis exhibits superiority to conventional multi-label learning algorithms and multi-label compatible neural networks.…”

Real-Time Pre-Identification and Cascaded Detection for Tiny Faces

“…Face detection is one of the most hot topics in computer vision as it is a key step for many different applications, such as face recognition [1], facial expression analysis [2], eye-tracking [3], facial performance capture [4], facial expression transformation [5], etc. In fact, the applications are not limited to the traditional areas, there are still some exciting interdisciplinary applications [6][7][8][9][10][11] in the field of animation.…”