Facial Affect Estimation in the Wild Using Deep Residual and Convolutional Networks

Hasani, Behzad; Mahoor, Mohammad H.

doi:10.1109/cvprw.2017.245

Cited by 17 publications

(9 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Arousal Valence CNN-M [19] 0.140 0.130 MM-Net [36] 0.088 0.134 FATAUVA [37] 0.095 0.123 DRC-Net [38] 0.094 0.161 PersEmoN (ours) 0.108 0.125 follows: 13) where N t denotes the total number of testing samples, Y P the ground truth, P i the prediction, andȲ P the average value of the ground truth.…”

Section: Methodsmentioning

confidence: 99%

PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship

Zhang

Peng

Winkler

2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Personality and emotion are both central to affective computing. Existing works solve them individually. In this paper we investigate if such high-level affect traits and their relationship can be jointly learned from face images in the wild. To this end, we introduce PersEmoN, an end-to-end trainable and deep Siamese-like network which we call emotion network and personality network, respectively. It consists of two convolutional network branches, one for emotion and the other for apparent personality. Both networks share their bottom feature extraction module and are optimized within a multi-task learning framework. Emotion and personality networks are dedicated to their own annotated dataset. An adversarial-like loss function is further employed to promote representation coherence among heterogeneous dataset sources. Based on this, the emotion-to-personality relationship is also well explored. Extensive experiments are provided to demonstrate the effectiveness of PersEmoN.

show abstract

Section: Methodsmentioning

confidence: 99%

PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship

Zhang

Peng

Winkler

2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

“…On the other hand, all traditional layers use a batch normalization layer, and ReLU is applied as an activation function to avoid the vanishing gradient problem. Moreover, based on the proposal by Behzad Hasani et al in [35], we changed all of the "valid" padding to "same" padding in order to reduce the size of the feature map, which means that all output grids will be the same size as their input grids. This model was designed using the Tensorflow libraries on an NVIDIA GeForce TI 1080 GPU with a learning rate of 0.0001.…”

Section: Inception-resnet V2 Model Architecturementioning

confidence: 99%

A Comparative Study of Deep CNN in Forecasting and Classifying the Macronutrient Deficiencies on Development of Tomato Plant

Tran

Choi

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

During the process of plant growth, such as during the flowering stages and fruit development, the plants need to be provided with the various minerals and nutrients to grow. Nutrient deficiency is the cause of serious diseases in plant growth, affecting crop yield. In this article, we employed artificial neural network models to recognize, classify, and predict the nutritional deficiencies occurring in tomato plants (Solanum lycopersicum L.). To classify and predict the different macronutrient deficiencies in the cropping process, this paper handles the captured images of the macronutrient deficiency. This deficiency during the fruiting and leafing phases of tomato plant are based on a deep convolutional neural network (CNN). A total of 571 images were captured with tomato leaves and fruits containing the crop species at the growth stage. Among all images, 80% (461 captured images) were used for the training dataset and 20% (110 captured images) were applied for the validation dataset. In this study, we provide an analysis of two different model architectures based on convolutional neural network for classifying and predicting the nutrient deficiency symptoms. For instance, Inception-ResNet v2 and Autoencoder with the captured images of tomato plant growth under the greenhouse conditions. Moreover, a major type of statistical structure, namely Ensemble Averaging, was applied with two aforementioned predictive models to increase the accuracy of predictive validation. Three mineral nutrients, i.e., Calcium/Ca2+, Potassium/K+, and Nitrogen/N, are considered for use in evaluating the nutrient status in the development of tomato plant with these models. The aim of this study is to predict the nutrient deficiency accurately in order to increase crop production and prevent the emergence of tomato pathology caused by lack of nutrients. The predictive performance of the three models in this paper are validated, with the accuracy rates of 87.273% and 79.091% for Inception-ResNet v2 and Autoencoder, respectively, and with 91% validity using Ensemble Averaging.

show abstract

“…In [24] a Bidirectional Long Short Term Memory Network (BLSTM) is used with hand-crafted features, showing better results than SVR. Hasani et al [12] extract features using Inception module [42]. Combining the it with an LSTM yields better results than using the Inception module only in a per-frame basis.…”

Section: Related Workmentioning

confidence: 99%

Dynamic Facial Models for Video-Based Dimensional Affect Estimation

Song

Sánchez-Lozano

Tellamekala

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Dimensional affect estimation from a face video is a challenging task, mainly due to the large number of possible facial displays made up of a set of behaviour primitives including facial muscle actions. The displays vary not only in composition but also in temporal evolution, with each display composed of behaviour primitives with varying in their short and long-term characteristics. Most existing work models affect relies on complex hierarchical recurrent models unable to capture short-term dynamics well. In this paper, we propose to encode these short-term facial shape and appearance dynamics in an image, where only the semantic meaningful information is encoded into the dynamic face images. We also propose binary dynamic facial masks to remove 'stable pixels' from the dynamic images. This process allows filtering of non-dynamic information, i.e. only pixels that have changed in the sequence are retained. Then, the final proposed Dynamic Facial Model (DFM) encodes both filtered facial appearance and shape dynamics of a image sequence preceding to the given frame into a threechannel raster image. A CNN-RNN architecture is tasked with modelling primarily the long-term changes. Experiments show that our dynamic face images achieved superior performance over the standard RGB face images on dimensional affect prediction task.

show abstract

Facial Affect Estimation in the Wild Using Deep Residual and Convolutional Networks

Cited by 17 publications

References 35 publications

PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship

PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship

A Comparative Study of Deep CNN in Forecasting and Classifying the Macronutrient Deficiencies on Development of Tomato Plant

Dynamic Facial Models for Video-Based Dimensional Affect Estimation

Contact Info

Product

Resources

About