Self-Paced Label Distribution Learning for In-The-Wild Facial Expression Recognition

Shao, Jianjian; Wu, Zhenqian; Luo, Yuanyan; Pu, Xiaorong; Ren, Yazhou

doi:10.1145/3503161.3547960

Cited by 14 publications

(5 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RAF-DB dataset is one of the most widely used large-scale realworld FER datasets because it facilitates fair comparisons, in which all images are cropped and do not require any additional preprocessing. Results show that our FaceFormer achieves state-of-the-art performance compared to all other methods, including FER with unconstrained variations (RAN [11], MA-Net [13], IPD-FER [58]), and FER with annotation ambiguity (SCN [30], DMUE [28], KTN [59], EfficientFace [26], SPLDL [29], EASE [32]). In particular, when compared to TransFER [23], the previous best achieved by combining CNN and ViT, FER-former lowers the error rate from 9.09% to 8.7%, a 4.3% improvement.…”

Section: A Comparison With State-of-the-art Methodsmentioning

confidence: 98%

“…RAN [11] 2020 86.90 SCN [30] 2020 88.14 DLN [60] 2021 86.40 KTN [59] 2021 88.07 MA-Net [13] 2021 88.40 DMUE [28] 2021 89.42 EfficientFace [26] 2021 88.36 TransFER [23] 2021 90.91 IPD-FER [58] 2022 88.89 CRS-CONT [61] 2022 88.07 SPLDL [29] 2022 89.08 EASE [32] 2022 89.56 FER-former (Ours) 2023 91.30…”

Section: Methods Years Acc(%)mentioning

confidence: 99%

“…Zhao et al [26] treat the output of an auxiliary ResNet-50 as probability distribution to guide the learning of the other backbone network. Shao et al [29] further adopt an auxiliary network as a label distribution generator to generate label distributions for guiding the backbone network training and selecting easy samples. Chen et al [27] put forward an approach to learn label distribution by calculating the confidence probability of its nearest neighbours.…”

Section: Related Workmentioning

confidence: 99%

“…Lots of efforts have been made to address the issue of annotation ambiguity and can be roughly categorized as two mainstream directions: Learning of Soft Label Distribution and Learning from Hard Samples via delicately-designed mechanisms (e.g., attention mechanism or self-supervised strategy). Specifically, i) Instead of hard one-hot supervisory signals, a set of probability distributions with auxiliary priors are assigned to facial images with the goal of driving the deep learning-based models to extract representations with fuzzy boundaries [26]- [29]; ii) Hard samples are selected as outliers and then imposed on extra special processes to suppress their negative impacts on the holistic models' learning [29]- [32]. Even though a series of attempts have been conducted to ease annotation ambiguity, the required auxiliary supervisory signals are still derived from raw one-hot labels and the ambiguity is prone to be inherited.…”

mentioning

confidence: 99%

See 3 more Smart Citations

FER-former: Multi-modal Transformer for Facial Expression Recognition

Li¹,

Wang²,

Gong³

et al. 2023

Preprint

View full text Add to dashboard Cite

The ever-increasing demands for intuitive interactions in Virtual Reality has triggered a boom in the realm of Facial Expression Recognition (FER). To address the limitations in existing approaches (e.g., narrow receptive fields and homogenous supervisory signals) and further cement the capacity of FER tools, a novel multifarious supervision-steering Transformer for FER in the wild is proposed in this paper. Referred as FERformer, our approach features multi-granularity embedding integration, hybrid self-attention scheme, and heterogeneous domainsteering supervision. In specific, to dig deep into the merits of the combination of features provided by prevailing CNNs and Transformers, a hybrid stem is designed to cascade two types of learning paradigms simultaneously. Wherein, a FER-specific transformer mechanism is devised to characterize conventional hard one-hot label-focusing and CLIP-based text-oriented tokens in parallel for final classification. To ease the issue of annotation ambiguity, a heterogeneous domains-steering supervision module is proposed to make image features also have text-space semantic correlations by supervising the similarity between image features and text features. On top of the collaboration of multifarious token heads, diverse global receptive fields with multi-modal semantic cues are captured, thereby delivering superb learning capability. Extensive experiments on popular benchmarks demonstrate the superiority of the proposed FER-former over the existing state-of-the-arts.

show abstract

Section: A Comparison With State-of-the-art Methodsmentioning

confidence: 98%

Section: Methods Years Acc(%)mentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

FER-former: Multi-modal Transformer for Facial Expression Recognition

Li¹,

Wang²,

Gong³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Therefore, data-driven methods will undoubtedly suffer from disturbances and get stuck into bad local minima during training, thereby remarkably degrading the performance. To tackle this problem, self-paced learning (SPL) (Kumar, Packer, and Koller 2010;Shao et al 2022;Huang et al 2021;Pan et al 2020) was proposed to train the model from 'easy' to 'hard' samples inspired by human cognitive learning, which has been proven to be beneficial in alleviating the noise/outlier problem (Li et al 2021). However, almost all CMH methods treat all instances and features equally during learning hash codes, while ignoring the difficulty differences caused by noise or outliers.…”

Section: Introductionmentioning

confidence: 99%

Dual Self-Paced Cross-Modal Hashing

Sun,

Dai,

Ren

et al. 2024

AAAI

View full text Add to dashboard Cite

Cross-modal hashing~(CMH) is an efficient technique to retrieve relevant data across different modalities, such as images, texts, and videos, which has attracted more and more attention due to its low storage cost and fast query speed. Although existing CMH methods achieve remarkable processes, almost all of them treat all samples of varying difficulty levels without discrimination, thus leaving them vulnerable to noise or outliers. Based on this observation, we reveal and study dual difficulty levels implied in cross-modal hashing learning, \ie instance-level and feature-level difficulty. To address this problem, we propose a novel Dual Self-Paced Cross-Modal Hashing (DSCMH) that mimics human cognitive learning to learn hashing from ``easy'' to ``hard'' in both instance and feature levels, thereby embracing robustness against noise/outliers. Specifically, our DSCMH assigns weights to each instance and feature to measure their difficulty or reliability, and then uses these weights to automatically filter out the noisy and irrelevant data points in the original space. By gradually increasing the weights during training, our method can focus on more instances and features from ``easy'' to ``hard'' in training, thus mitigating the adverse effects of noise or outliers. Extensive experiments are conducted on three widely-used benchmark datasets to demonstrate the effectiveness and robustness of the proposed DSCMH over 12 state-of-the-art CMH methods.

show abstract

Label distribution feature selection based on neighborhood rough set

Wu,

Guo,

Lin

2024

Concurrency and Computation

View full text Add to dashboard Cite

SummaryIn label distribution learning (LDL), an instance is involved with many labels in different importance degrees, and the feature space of instances is accompanied with thousands of redundant and/or irrelevant features. Therefore, the main characteristic of feature selection in LDL is to evaluate the ability of each feature. Motivated by neighborhood rough set (NRS), which can be used to measure the dependency degree of feature via constructing neighborhood relations on feature space and label space, respectively, this article proposes a novel label distribution feature selection method. In this article, the neighborhood class of instance in label distribution space is defined, which is beneficial to recognize the logical class of target instance. Then, a new NRS model for LDL is proposed. Specially, the dependency degree of feature combining label weight is defined. Finally, a label distribution feature selection based on NRS is presented. Extensive experiments on 12 data sets show the effectiveness of the proposed algorithm.

show abstract

Self-Paced Label Distribution Learning for In-The-Wild Facial Expression Recognition

Cited by 14 publications

References 36 publications

FER-former: Multi-modal Transformer for Facial Expression Recognition

FER-former: Multi-modal Transformer for Facial Expression Recognition

Dual Self-Paced Cross-Modal Hashing

Label distribution feature selection based on neighborhood rough set

Contact Info

Product

Resources

About