Deep Learning for Spatio-Temporal Modeling of Dynamic Spontaneous Emotions

Chanti, Dawood Al; Caplier, Alice

doi:10.1109/taffc.2018.2873600

Cited by 31 publications

(16 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, Yun et al [33] present an engagement detection method processing facial videos with a CNN architecture that includes a layer modeling the temporal long and short-term data dynamics. Chanti et al [34] use the combination of 3D-CNN to model short-term spatio-temporal features and Convolutional-LSTM to learn global spatio-temporal features for video-based facial expression analysis. Similarly, a combination of CNN and LSTM models are recently tested for affect recognition from audio and video facial expressions data [35], [36].…”

Section: Modeling Multiple Temporal Scalesmentioning

confidence: 99%

Modeling Multiple Temporal Scales of Full-Body Movements for Emotion Classification

Beyan

Karumuri

Volpe

et al. 2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

This work investigates classification of emotions from full-body movements by using a novel Convolutional Neural Network-based architecture. The model is composed of two shallow networks processing in parallel when the 8-bit RGB images obtained from time intervals of 3D-positional data are the inputs. One network performs a coarse-grained modelling in the time domain while the other one applies a fine-grained modelling. We show that combining different temporal scales into a single architecture improves the classification results of a dataset composed of short excerpts of the performances of professional dancers who interpreted four affective states: anger, happiness, sadness, and insecurity. Additionally, we investigate the effect of data chunk duration, overlapping, the size of the input images and the contribution of several data augmentation strategies for our proposed method. Better recognition results were obtained when the duration of a data chunk was longer, and this was further improved by applying balanced data augmentation. Moreover, we test our method on other existing motion capture datasets and compare the results with prior art. In all experiments, our results surpassed the state-of-the-art approaches, showing that this method generalizes across diverse settings and contexts.

show abstract

Section: Modeling Multiple Temporal Scalesmentioning

confidence: 99%

Modeling Multiple Temporal Scales of Full-Body Movements for Emotion Classification

Beyan

Karumuri

Volpe

et al. 2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

“…Chanti et.al. applied grid convolution to encoder the spatial correlation and used LSTM to model the temporal relationship [2] . In another work, Tran el.…”

Section: Related Workmentioning

confidence: 99%

Progressive ShallowNet for large scale dynamic and spontaneous facial behaviour analysis in children

Qayyum

Razzak

Moustafa

et al. 2022

Image and Vision Computing

View full text Add to dashboard Cite

COVID-19 has severely disrupted every aspect of society and left negative impact on our life. Resisting the temptation in engaging face-to-face social connection is not as easy as we imagine. Breaking ties within social circle makes us lonely and isolated, that in turns increase the likelihood of depression related disease and even can leads to death by increasing the chance of heart disease. Not only adults, children's are equally impacted where the contribution of emotional competence to social competence has long term implications. Early identification skill for facial behaviour emotions, deficits, and expression may help to prevent the low social functioning. Deficits in young children's ability to differentiate human emotions can leads to social functioning impairment. However, the existing work focus on adult emotions recognition mostly and ignores emotion recognition in children. By considering the working of pyramidal cells in the cerebral cortex, in this paper, we present progressive lightweight shallow learning for the classification by efficiently utilizing the skip-connection for spontaneous facial behaviour recognition in children. Unlike earlier deep neural networks, we limit the alternative path for the gradient at the earlier part of the network by increase gradually with the depth of the network. Progressive ShallowNet is not only able to explore more feature space but also resolve the over-fitting issue for smaller data, due to limiting the residual path locally, making the network vulnerable to perturbations. We have conducted extensive experiments on on benchmark facial behaviour analysis in children that showed significant performance gain comparatively.

show abstract

“…Wu et al discussed the deep local video feature (DLVF) method based on the TSN method, assigning different weights to different segments in TSN and fusing the judgment results of different segments more reasonably [ 21 ]. Chanti et al envisioned a behavior recognition method based on temporal and spatial weighted gesture motion characteristics through dynamic time warping and Fourier time pyramid algorithm modeling, and finally improved the accuracy of human behavior recognition [ 22 ].…”

Section: Related Workmentioning

confidence: 99%

Data Feature Extraction Method of Wearable Sensor Based on Convolutional Neural Network

Wang¹

2022

Journal of Healthcare Engineering

View full text Add to dashboard Cite

With the rapid development of society and science technology, human health issues have attracted much attention due to wearable devices’ ability to provide high-quality sports, health, and activity monitoring services. This paper proposes a method for feature extraction of wearable sensor data based on a convolutional neural network (CNN). First, it uses the Kalman filter to fuse the data to obtain a preliminary state estimation, and then it uses CNN to recognize human behavior, thereby obtaining the corresponding behavior set. Moreover, this paper conducts experiments on 5 datasets. The experimental results show that the method in this paper extracts data features at multiple scales while fully maintaining data independence, can effectively extract corresponding feature data, and has strong generalization ability, which can adapt to different learning tasks.

show abstract

Deep Learning for Spatio-Temporal Modeling of Dynamic Spontaneous Emotions

Cited by 31 publications

References 38 publications

Modeling Multiple Temporal Scales of Full-Body Movements for Emotion Classification

Modeling Multiple Temporal Scales of Full-Body Movements for Emotion Classification

Progressive ShallowNet for large scale dynamic and spontaneous facial behaviour analysis in children

Data Feature Extraction Method of Wearable Sensor Based on Convolutional Neural Network

Contact Info

Product

Resources

About