Cross-Database Facial Expression Recognition Based on Fine-Tuned Deep Convolutional Network

Zavarez, Marcus Vinicius; Berriel, Rodrigo F.; Oliveira-Santos, Thiago

doi:10.1109/sibgrapi.2017.60

Cited by 76 publications

(52 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For CK+ dataset, Our ECAN achieves much better performance than most other methods except [47] that conducted cross-dataset facial expression recognition combining six labcontrolled datasets as the source domain. It is worth noting that CK+ and these lab-controlled datasets are very similar in many respects, such as the controlled collection environment, subject characters, illumination condition and head postures.…”

Section: Resultsmentioning

confidence: 94%

“…So we further evaluate our method using RAF-DB 2.0 with other three lab-controlled datasets as our multisource domain, and achieve the best cross-dataset performance 89.69%. On the other hand, when compared to the CK+ indataset results in Table III For JAFFE dataset, which is a highly biased dataset in respect of gender and ethnicity, i.e., it only contains ten Japanese females, the fine-tuning technique used in [47] which reported high accuracy on CK+ dataset is no longer effective in this context. In contrast, by matching the marginal and conditional distribution and also the class distribution across domains, our method yields the best performance and is For MMI dataset, our ECAN achieves 69.89% cross-dataset accuracy, which outperforms all the other compared methods and also the in-dataset performance shown in Table III-B. Comparing with the baselines, we can find that the original CNN structure is inferior than some previous methods that use more similar source dataset (such as CK+) with target MMI dataset and the original MMD (CNN+MMD) only gains a negligible improvement.…”

Section: Resultsmentioning

confidence: 97%

See 1 more Smart Citation

A Deeper Look at Facial Expression Dataset Bias

Deng

2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Datasets play an important role in the progress of facial expression recognition algorithms, but they may suffer from obvious biases caused by different cultures and collection conditions. To look deeper into this bias, we first conduct comprehensive experiments on dataset recognition and crossdataset generalization tasks, and for the first time explore the intrinsic causes of the dataset discrepancy. The results quantitatively verify that current datasets have a strong buildin bias and corresponding analyses indicate that the conditional probability distributions between source and target datasets are different. However, previous researches are mainly based on shallow features with limited discriminative ability under the assumption that the conditional distribution remains unchanged across domains. To address these issues, we further propose a novel deep Emotion-Conditional Adaption Network (ECAN) to learn domain-invariant and discriminative feature representations, which can match both the marginal and the conditional distributions across domains simultaneously. In addition, the largely ignored expression class distribution bias is also addressed by a learnable re-weighting parameter, so that the training and testing domains can share similar class distribution. Extensive cross-database experiments on both lab-controlled datasets (CK+, JAFFE, MMI and Oulu-CASIA) and real-world databases (Af-fectNet, FER2013, RAF-DB 2.0 and SFEW 2.0) demonstrate that our ECAN can yield competitive performances across various facial expression transfer tasks and outperform the state-of-theart methods.

show abstract

Section: Resultsmentioning

confidence: 94%

Section: Resultsmentioning

confidence: 97%

A Deeper Look at Facial Expression Dataset Bias

Deng

2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

“…have been shown to be very effective. However, the most recent approaches are based on deep convolutional neural networks (CNNs) [9], [11], [13], [17], [20], [21], [22], [23]. However, one of the main drawbacks is the amount of data required to train such deep networks.…”

Section: Introductionmentioning

confidence: 99%

Memory Integrity of CNNs for Cross-Dataset Facial Expression Recognition

Tannugi

Britto

Koerich

2019

2019 IEEE International Conference on Systems, Man and Cybernetics (SMC)

View full text Add to dashboard Cite

Facial expression recognition is a major problem in the domain of artificial intelligence. One of the best ways to solve this problem is the use of convolutional neural networks (CNNs). However, a large amount of data is required to train properly these networks but most of the datasets available for facial expression recognition are relatively small. A common way to circumvent the lack of data is to use CNNs trained on large datasets of different domains and fine-tuning the layers of such networks to the target domain. However, the fine-tuning process does not preserve the memory integrity as CNNs have the tendency to forget patterns they have learned. In this paper, we evaluate different strategies of fine-tuning a CNN with the aim of assessing the memory integrity of such strategies in a cross-dataset scenario. A CNN pre-trained on a source dataset is used as the baseline and four adaptation strategies have been evaluated: fine-tuning its fully connected layers; fine-tuning its last convolutional layer and its fully connected layers; retraining the CNN on a target dataset; and the fusion of the source and target datasets and retraining the CNN. Experimental results on four datasets have shown that the fusion of the source and the target datasets provides the best trade-off between accuracy and memory integrity.

show abstract

“…This leads to a challenging problem, namely, the cross-database nonfrontal FER problem. To cope with this challenge, many effective approaches such as subspace-based methods and based on deep learning models had been proposed in recent years [16], [17], [18] [19], [20], [21], [22], [23]. To leverage the distribution discrepancy between training and testing facial expression images, in our preliminary work in [19], zheng et al proposed a novel transductive transfer subspace learning method to jointly learn a discriminative subspace and to predict the label values of the unlabelled facial expression images by using all labelled training samples from source domain and an unlabelled auxiliary testing samples set from target domain.…”

mentioning

confidence: 99%

“…In [20], Wei et al proposed a deep nonlinear feature coding framework for unsupervised cross-domain FER problem, which introduce domain divergence minimization by Maximum Mean Discrepancy (MMD) and kernelization coding to build on a marginalized stacked denoising auto-encoder for extracting very efficient deep features. Zavarez et al [21] utilize the fine-tune trick in deep convolutional network for cross-database video-based FER problem in several well-established facial expression databases. However, these methods of cross-database FER are typically based on the frontal and near-frontal facial samples or the samples of each domain with single view-point in their experiments.…”

mentioning

confidence: 99%

Cross-Domain Facial Expression Recognition Based on Transductive Deep Transfer Learning

et al. 2019

View full text Add to dashboard Cite

Cross-database non-frontal expression recognition is a very meaningful but rather difficult subject in the fields of computer vision and affect computing. In this paper, we proposed a novel transductive deep transfer learning architecture based on widely used VGGface16-Net for this problem. In this framework, the VGGface16-Net is used to jointly learn an common optimal nonlinear discriminative features from the non-frontal facial expression samples between the source and target databases and then we design a novel transductive transfer layer to deal with the cross-database non-frontal facial expression classification task. In order to validate the performance of the proposed transductive deep transfer learning networks, we present extensive crossdatabase experiments on two famous available facial expression databases, namely the BU-3DEF and the Multi-PIE database. The final experimental results show that our transductive deep transfer network outperforms the state-of-the-art cross-database facial expression recognition methods.

show abstract

Cross-Database Facial Expression Recognition Based on Fine-Tuned Deep Convolutional Network

Cited by 76 publications

References 22 publications

A Deeper Look at Facial Expression Dataset Bias

A Deeper Look at Facial Expression Dataset Bias

Memory Integrity of CNNs for Cross-Dataset Facial Expression Recognition

Cross-Domain Facial Expression Recognition Based on Transductive Deep Transfer Learning

Contact Info

Product

Resources

About