From Facial Expression Recognition to Interpersonal Relation Prediction

Zhang, Zhanpeng; Luo, Ping; Loy, Chen Change; Tang, Xiaoou

doi:10.1007/s11263-017-1055-1

Cited by 240 publications

(119 citation statements)

References 82 publications

Supporting

Mentioning

118

Contrasting

Order By: Relevance

“…Training Settings. We split the datasets (CEW [46], CelebA [45], EmotioNet [43], ExpW [44]) into 75% for training, 10% for validation, and 15% for testing. Many of the detected expressions have a high ratio of negative to positive examples (i.e.…”

Section: Classification Of Facial Expressionsmentioning

confidence: 99%

“…In order to avoid the biasing of the classifier to the most frequent class (negative class), the positive and negative examples are balanced in the training set by undersampling [47]. The ExpW [44] dataset is annotated for 6 emotional expressions and the neutral expression. In order to keep the training set balanced and diverse when training for the detection of the neutral expression, negative examples equal to positive examples are drawn from all the 6 emotional expressions.…”

Section: Classification Of Facial Expressionsmentioning

confidence: 99%

“…We observe that the performance is highly dependent on the number of training samples and the variance in expression-appearance. More specifically, expressions like lips part, and eyes closed have TABLE 1 The classification results obtained by the frame-based components of the network proposed in [18] and by the proposed AFEA method over different facial expressions on the 15% testing splits of the EmotioNet [43], ExpW [44], CelebA [45], and CEW [46] On other expressions like brow lowerer, and lid tightener we obtain moderately good performance due to the large variance in expression-appearance among different people. Finally, we obtain low F1-score values for the outer brow raiser and the upper lid raiser as the EmotioNet dataset is highly imbalanced for those two classes.…”

Section: Classification Of Facial Expressionsmentioning

confidence: 99%

See 2 more Smart Citations

SchiNet: Automatic Estimation of Symptoms of Schizophrenia from Facial Behaviour Analysis

Bishay

Palasek²,

Priebe

et al. 2021

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Patients with schizophrenia often display impairments in the expression of emotion and speech and those are observed in their facial behaviour. Automatic analysis of patients' facial expressions that is aimed at estimating symptoms of schizophrenia has received attention recently. However, the datasets that are typically used for training and evaluating the developed methods, contain only a small number of patients (4-34) and are recorded while the subjects were performing controlled tasks such as listening to life vignettes, or answering emotional questions. In this paper, we use videos of professional-patient interviews, in which symptoms were assessed in a standardised way as they should/may be assessed in practice, and which were recorded in realistic conditions (i.e. varying illumination levels and camera viewpoints) at the patients' homes or at mental health services. We automatically analyse the facial behaviour of 91 out-patients -this is almost 3 times the number of patients in other studies -and propose SchiNet, a novel neural network architecture that estimates expression-related symptoms in two different assessment interviews. We evaluate the proposed SchiNet for patient-independent prediction of symptoms of schizophrenia. Experimental results show that some automatically detected facial expressions are significantly correlated to symptoms of schizophrenia, and that the proposed network for estimating symptom severity delivers promising results.

show abstract

Section: Classification Of Facial Expressionsmentioning

confidence: 99%

Section: Classification Of Facial Expressionsmentioning

confidence: 99%

Section: Classification Of Facial Expressionsmentioning

confidence: 99%

See 1 more Smart Citation

SchiNet: Automatic Estimation of Symptoms of Schizophrenia from Facial Behaviour Analysis

Bishay

Palasek²,

Priebe

et al. 2021

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

“…Expression datasets: Several facial expression datasets have been created in the past that consist of face images labeled with discrete emotion categories [4,9,10,11,16,17,31,34,40,41,43,54,55], facial action units [4,34,36,37,43], and strengths of valence and arousal [25,27,28,40,44]. While these datasets played a significant role in the advancement of automatic facial expression analysis in terms of emotion recognition, action unit detection and valence-arousal estimation, they are not the best fit for learning a compact expression embedding space that mimics human visual preferences.…”

Section: Related Workmentioning

confidence: 99%

A Compact Embedding for Facial Expression Similarity

Vemulapalli

Agarwala

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Most of the existing work on automatic facial expression analysis focuses on discrete emotion recognition, or facial action unit detection. However, facial expressions do not always fall neatly into pre-defined semantic categories. Also, the similarity between expressions measured in the action unit space need not correspond to how humans perceive expression similarity. Different from previous work, our goal is to describe facial expressions in a continuous fashion using a compact embedding space that mimics human visual preferences. To achieve this goal, we collect a large-scale faces-in-the-wild dataset with human annotations in the form: Expressions A and B are visually more similar when compared to expression C, and use this dataset to train a neural network that produces a compact (16-dimensional) expression embedding. We experimentally demonstrate that the learned embedding can be successfully used for various applications such as expression retrieval, photo album summarization, and emotion recognition. We also show that the embedding learned using the proposed dataset performs better than several other embeddings learned using existing emotion or action unit datasets.

show abstract

“…The interdisciplinary research of multimedia and sociology has been studied for many years [2,12]. Popular topics include social networks discovery [13], key actors detection [14], group activity recognition [15], and so on.…”

Section: Related Workmentioning

confidence: 99%

Multi-Granularity Reasoning for Social Relation Recognition From Images

Zhang

Liu

et al. 2019

2019 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

Discovering social relations in images can make machines better interpret the behavior of human beings. However, automatically recognizing social relations in images is a challenging task due to the significant gap between the domains of visual content and social relation. Existing studies separately process various features such as faces expressions, body appearance, and contextual objects, thus they cannot comprehensively capture the multi-granularity semantics, such as scenes, regional cues of persons, and interactions among persons and objects. To bridge the domain gap, we propose a Multi-Granularity Reasoning framework for social relation recognition from images. The global knowledge and midlevel details are learned from the whole scene and the regions of persons and objects, respectively. Most importantly, we explore the fine-granularity pose keypoints of persons to discover the interactions among persons and objects. Specifically, the pose-guided Person-Object Graph and Person-Pose Graph are proposed to model the actions from persons to object and the interactions between paired persons, respectively. Based on the graphs, social relation reasoning is performed by graph convolutional networks. Finally, the global features and reasoned knowledge are integrated as a comprehensive representation for social relation recognition. Extensive experiments on two public datasets show the effectiveness of the proposed framework.

show abstract

From Facial Expression Recognition to Interpersonal Relation Prediction

Cited by 240 publications

References 82 publications

SchiNet: Automatic Estimation of Symptoms of Schizophrenia from Facial Behaviour Analysis

SchiNet: Automatic Estimation of Symptoms of Schizophrenia from Facial Behaviour Analysis

A Compact Embedding for Facial Expression Similarity

Multi-Granularity Reasoning for Social Relation Recognition From Images

Contact Info

Product

Resources

About