Kernelized Dense Layers For Facial Expression Recognition

International Conference on Content-Based Multimedia Indexing

et al. 2022

Self Cite

Cultural heritage understanding and preservation is an important issue for society as it represents a fundamental aspect of its identity. Paintings represent a significant part of cultural heritage, and are the subject of study continuously. However, the way viewers perceive paintings is strictly related to the so-called HVS (Human Vision System) behaviour. This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. In further details, we introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans, including the fundamental understanding of a scene, and then extend it to painting images. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention. We use an FCNN (Fully Convolutional Neural Network), in which we exploit a differentiable channel-wise selection and Soft-Argmax modules. We also incorporate learnable Gaussian distributions onto the network bottleneck to simulate visual attention process bias in natural scene images. Furthermore, to reduce the effect of shifts between different domains (i.e. natural images, painting), we urge the model to learn unsupervised general features from other domains using a gradient reversal classifier. The results obtained by our model outperform existing state-of-the-art ones in terms of accuracy and efficiency.

Section: Feature Extractormentioning

confidence: 99%

A domain adaptive deep learning solution for scanpath prediction of paintings

Kerkouri

Tliba

International Conference on Content-Based Multimedia Indexing

et al. 2022

Self Cite

“…As reported by [2,11,10], these kernels fail to learn fully linearly separable features, despite of being simple and computationally inexpensive. Whereas, higher order kernel functions are able to better fit input data with additional computation.…”

Section: Taylor Series Kernelized Layermentioning

confidence: 99%

“…First of all, to expand the dense layer, we use the method of kernelized Dense Layer (KDL), proposed in [11]. KDL is similar to a classical neuron layer in the way that it applies a dot product between a vector of weights and an input vector, adds a bias vector (b ≥ 0) and eventually applies an activation function.…”

Section: Taylor Series Kernelized Layermentioning

confidence: 99%

“…It leverages kernel functions which allow to generalise linear pooling while capturing higher order information. Mahmoudi et al [11] also introduced a novel dense layer based on kernel function. It applies a higher order function on its inputs instead of calculating their weighted sum.…”

Section: Introductionmentioning

confidence: 99%

“…The proposed Kernelized Dense Layer permits to improve the discrimination power of the full network and it is completely differentiable, allowing an end-to-end learning. Similarly to [11], Jayasumana et al [2] proposed a kernelized classification layer for deep networks. This method is similar to the Kernelized Dense Layers, yet it is only used on the last (softmax) layer.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Taylor Series Kernelized Layer for Fine-Grained Recognition

Mahmoudi

2021 IEEE International Conference on Image Processing (ICIP)

Boufera

et al. 2021

Self Cite

In this paper, we propose a new architecture to enhance dense layers with a Taylor Series Kernelized Layer (TSKL). The proposed layer expands the underlying linear kernel of dense layers to a higher-order Taylor series kernel. This kernel is able to learn more complex patterns than the linear one and thus be more discriminative. In other words, TKSL first maps input data to a higher-dimensional Reproducing Kernel Hilbert Space (RKHS). After that, it learns a linear classifier in that RKHS which corresponds to a powerful non-linear classifier in the original feature space. The mapping features to a higher-order RKHS is performed implicitly by leveraging the kernel trick and explicitly by combining multiple kernels. The experimental results demonstrate that the proposed layer outperforms the ordinary dense layer when uses in both Multilayer Perceptrons (MLPs) and Convolutional Neural Networks (CNNs).

Improved Bilinear Model for Facial Expression Recognition

Mahmoudi

Pattern Recognition and Artificial Intelligence

Boufera

et al. 2021

Self Cite

Facial Expression Recognition (FER) systems aims to classify human emotions through facial expression as one of seven basic emotions: happiness, sadness, fear, disgust, anger, surprise and neutral. FER is a very challenging problem due to the subtle differences that exist between its categories. Even though convolutional neural networks (CNN) achieved impressive results in several computer vision tasks, they still do not perform as well in FER. Many techniques, like bilinear pooling and improved bilinear pooling, have been proposed to improve the CNN performance on similar problems. The accuracy enhancement they brought in multiple visual tasks, shows that their is still room for improvement for CNNs on FER. In this paper, we propose to use bilinear and improved bilinear pooling with CNNs for FER. This framework has been evaluated on three well known datasets, namely ExpW, FER2013 and RAF-DB. It has shown that the use of bilinear and improved bilinear pooling with CNNs can enhance the overall accuracy to nearly 3% for FER and achieve state-of-the-art results.