Learnable pooling weights for facial expression recognition

Mahmoudi, M. Amine; Chetouani, Aladine; Boufera, Fatma; Tabia, Hedi

doi:10.1016/j.patrec.2020.09.001

Cited by 18 publications

(5 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CNNs are used to model their features before extracting them. Different architectures were proposed and evolved for this end [27,28]. Here, we use MobileNet [32] to extract features from the input images.…”

Section: Feature Extractormentioning

confidence: 99%

A domain adaptive deep learning solution for scanpath prediction of paintings

Kerkouri

Tliba

Chetouani

et al. 2022

International Conference on Content-Based Multimedia Indexing

Self Cite

View full text Add to dashboard Cite

Cultural heritage understanding and preservation is an important issue for society as it represents a fundamental aspect of its identity. Paintings represent a significant part of cultural heritage, and are the subject of study continuously. However, the way viewers perceive paintings is strictly related to the so-called HVS (Human Vision System) behaviour. This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. In further details, we introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans, including the fundamental understanding of a scene, and then extend it to painting images. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention. We use an FCNN (Fully Convolutional Neural Network), in which we exploit a differentiable channel-wise selection and Soft-Argmax modules. We also incorporate learnable Gaussian distributions onto the network bottleneck to simulate visual attention process bias in natural scene images. Furthermore, to reduce the effect of shifts between different domains (i.e. natural images, painting), we urge the model to learn unsupervised general features from other domains using a gradient reversal classifier. The results obtained by our model outperform existing state-of-the-art ones in terms of accuracy and efficiency.

show abstract

Section: Feature Extractormentioning

confidence: 99%

A domain adaptive deep learning solution for scanpath prediction of paintings

Kerkouri

Tliba

Chetouani

et al. 2022

International Conference on Content-Based Multimedia Indexing

Self Cite

View full text Add to dashboard Cite

show abstract

“…For the rest of this work, we will refer to the models as {backbone name}-BB (e.g VGG-BB, ...). It is worth noting that sophisticated layers can be added to improve the results like learnable pooling [44], Kernelized Dense layer [43,45].…”

Section: Backbonementioning

confidence: 99%

A simple and efficient deep scanpath prediction

Kerkouri¹,

Chetouani²

2022

Self Cite

View full text Add to dashboard Cite

Visual scanpath is the sequence of fixation points that the human gaze travels while observing an image, and its prediction helps in modeling the visual attention of an image. To this end, several models were proposed in the literature using complex deep learning architectures and frameworks. Here, we explore the efficiency of using common deep learning architectures, in a simple fully convolutional regressive manner. We experiment with how well these models can predict the scanpaths on 2 datasets. We compare with other models using different metrics and show competitive results that sometimes surpass previous complex architectures. We also compare the different leveraged backbone architectures based on their performances in the experiment to deduce which ones are the most suitable for the task.

show abstract

“…Consequently, unlike the traditional approaches [36,40,41] which rely on constant and manually selected weights, we let the network learn the most suitable weights. This way, the networks not only learns to compute appropriate weights, but it also produces effective results which can be adapted for works where the need of manuallyinjected weights is discouraged [42,43].…”

Section: Trainable Multi-view Weightsmentioning

confidence: 99%

Disentangling 3D/4D Facial Affect Recognition With Faster Multi-View Transformer

Behzad

Zhao

2021

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

In this paper, we propose MiT: a novel multi-view transformer model 1 for 3D/4D facial affect recognition. MiT incorporates patch and position embeddings from various patches of multi-views and uses them for learning various facial muscle movements to showcase an effective recognition performance. We also propose a multi-view loss function that is not only gradient-friendly, and hence speeds up the gradient computation during back-propagation, but it also leverages the correlation associated with the underlying facial patterns among multiviews. Additionally, we offer multi-view weights that are trainable and learnable, and help substantially in training. Finally, we equip our model with distributed performance for faster learning and computational convenience. With the help of extensive experiments, we show that our model outperform the existing methods on widely-used datasets for 3D/4D FER.

show abstract

Learnable pooling weights for facial expression recognition

Cited by 18 publications

References 28 publications

A domain adaptive deep learning solution for scanpath prediction of paintings

A domain adaptive deep learning solution for scanpath prediction of paintings

A simple and efficient deep scanpath prediction

Disentangling 3D/4D Facial Affect Recognition With Faster Multi-View Transformer

Contact Info

Product

Resources

About