BReG-NeXt: Facial Affect Computing Using Adaptive Residual Networks With Bounded Gradient

Hasani, Behzad; Negi, Pooran; Mahoor, Mohammad H.

doi:10.1109/taffc.2020.2986440

Cited by 30 publications

(25 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We achieve the average accuracy of 79.01%, which is the highest among the methods that have reported this metric. On FER-2013 [42] dataset, our model achieves the classification accuracy of 72.03%, which outperforms the highest previous reported accuracy of 71.53%, using BReG-NeXt [13] by a margin of 0.5%. Fig.…”

Section: Classification Results and Comparisonmentioning

confidence: 67%

“…Proposing new CNNs is another widely used method for FER which is studied in the following researches: Hasani et al [13] proposed a CNN architecture using a function with bounded derivative instead of a simple shortcut path in the residual units for automatic recognition of facial expres-sions. Yu et al [14] proposed a multi-task framework for the global-local representation of facial expressions, where a shallow module is responsible for extracting information from local regions and the global image, and then a partbased module process the critical local regions.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild

Fard

Mahoor

2022

IEEE Access

Self Cite

View full text Add to dashboard Cite

Automated Facial Expression Recognition (FER) in the wild using deep neural networks is still challenging due to intra-class variations and inter-class similarities in facial images. Deep Metric Learning (DML) is among the widely used methods to deal with these issues by improving the discriminative power of the learned embedded features. This paper proposes an Adaptive Correlation (Ad-Corre) Loss to guide the network towards generating embedded feature vectors with high correlation for within-class samples and less correlation for between-class samples. Ad-Corre consists of 3 components called Feature Discriminator, Mean Discriminator, and Embedding Discriminator. We design the Feature Discriminator component to guide the network to create the embedded feature vectors to be highly correlated if they belong to a similar class, and less correlated if they belong to different classes. In addition, the Mean Discriminator component leads the network to make the mean embedded feature vectors of different classes to be less similar to each other. We use Xception network as the backbone of our model, and contrary to previous work, we propose an embedding feature space that contains k feature vectors. Then, the Embedding Discriminator component penalizes the network to generate the embedded feature vectors, which are dissimilar. We trained our model using the combination of our proposed loss functions called Ad-Corre Loss jointly with the cross-entropy loss. We achieved a very promising recognition accuracy on AffectNet, RAF-DB, and FER-2013. Our extensive experiments and ablation study indicate the power of our method to cope well with challenging FER tasks in the wild.

show abstract

Section: Classification Results and Comparisonmentioning

confidence: 67%

Section: Related Workmentioning

confidence: 99%

Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild

Fard

Mahoor

2022

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the first iteration, a random tree is generated by assigning random weights to the edges of the k 50 . We observe that as training is being performed, face [11] CNN 58 Hua et al [12] Ensemble 62.11 Chen et al [4] Facial mask 61.50 Kollias et al [17] Augmentation 60 Hasani et al [9] BreG-NeXt32 66.74 Hasani et al [9] BreG-NeXt50 trees tend to resemble the structure of human faces more and more. As an obvious example, face trees formed in iteration 40 for both AffectNet and FER2013 dataset closely resemble human faces around the jaw line, mouth, nose, and eyes, while still being customized for each dataset.…”

Section: B Tree Topology Learningmentioning

confidence: 98%

“…For AffectNet, the largest confusions occur for contempt-happy (where contempt is the true class and happy is the predicted class), sad-surprise, and neutral-happy cases. Confusing contempt with happy is common in FER systems, since it is very hard to distinguish between these two even for humans [9]. In terms of FER2013, the largest confusion occurs in the disgust-happy case.…”

Section: A Performancementioning

confidence: 99%

“…By altering the standard skip connections in a ResNet model with a differentiable function, the Bounded Residual Gradient Network (BReG-Net) was proposed in [10]. By adding trainable parameters to BReG-Net, the same authors later proposed BReG-NeXt [9], which offers state-of-the-art results on AffectNet [26] and FER2013 [7] datasets. More recently, self-supervised and contrastive approaches have also begun to gain popularity in this area, but have in many cases used either videos [28] or multi-view [31] data as opposed to the work presented in this study which focuses on a single image.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Face Trees for Expression Recognition

Kolahdouzi¹,

Sepas-Moghaddam²,

Etemad³

2021

Preprint

View full text Add to dashboard Cite

We propose an end-to-end architecture for facial expression recognition. Our model learns an optimal tree topology for facial landmarks, whose traversal generates a sequence from which we obtain an embedding to feed a sequential learner. The proposed architecture incorporates two main streams, one focusing on landmark positions to learn the structure of the face, while the other focuses on patches around the landmarks to learn texture information. Each stream is followed by an attention mechanism and the outputs are fed to a two-stream fusion component to perform the final classification. We conduct extensive experiments on two largescale publicly available facial expression datasets, AffectNet and FER2013, to evaluate the efficacy of our approach. Our method outperforms other solutions in the area and sets new state-ofthe-art expression recognition rates on these datasets.

show abstract

Emotion-aware Multi-view Contrastive Learning for Facial Emotion Recognition

Kim

Song

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

BReG-NeXt: Facial Affect Computing Using Adaptive Residual Networks With Bounded Gradient

Cited by 30 publications

References 49 publications

Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild

Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild

Face Trees for Expression Recognition

Emotion-aware Multi-view Contrastive Learning for Facial Emotion Recognition

Contact Info

Product

Resources

About