Action Units recognition based on Deep Spatial-Convolutional and Multi-label Residual network

Wang, Sujing; Lin, Bo; Wang, Yong; Yi, Tongqiang; Zou, Bochao; Lyu, Xiangwen

doi:10.1016/j.neucom.2019.05.018

Cited by 8 publications

(4 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [32] the authors proposed a unified framework for multilabel image classification, which uses CNNs and recurrent neural networks to model the label co-occurrence dependency in a joint image/label embedding space. Many other MLC problems have been addressed with CNNs [41,27,7,33,40,48,22]. However, most of these solutions are aimed at image and natural language processing.…”

Section: Deep Learning Methods For Mlc Problemsmentioning

confidence: 99%

Deep neural network to extract high-level features and labels in multi-label classification problems

et al. 2020

View full text Add to dashboard Cite

Section: Deep Learning Methods For Mlc Problemsmentioning

confidence: 99%

Deep neural network to extract high-level features and labels in multi-label classification problems

et al. 2020

View full text Add to dashboard Cite

“…J ÂA-Net is the latest research result and an improved version of pioneering work based on attention. On the CK+ dataset, it is compared with the recent related work BGCS [16], HRBM [18], JPML [15], DSCMR [22].…”

Section: Compared Methodsmentioning

confidence: 99%

“…Secondly, all the features are connected and sent into the full connection layer to learn the relationship of AUs and obtain the global features across sub-regions for AU detection. Wang et al [22] constructed a network using local convolution and residual units and proposed a slope loss function for multi-label learning. Ma et al [7] proposed a method to segment human face regions into several regions using expert prior knowledge.…”

Section: Related Workmentioning

confidence: 99%

Computational efficient deep neural network with difference attention maps for facial action unit detection

Chen,

Wang,

Wang

et al. 2020

Preprint

View full text Add to dashboard Cite

There are more and more applications of attention mechanism in the field of computer vision. As a task in the field of vision, facial action unit detection has gradually used attention mechanism. Attention mechanism can make the network pay more attention to specific areas to capture the features related to AU. Most of the existing attention-based methods use facial landmark detection to predefine the regions associated with AU. These methods depend on the accuracy of facial landmark detection. Region learning is not conducive to learning the correlation between AUs. The number of predefined regions of interest is generally fixed. Processing parts that have nothing to do with the target task also increases the computational burden and network complexity. In this paper, we propose a computational efficient end-to-end training deep neural network (CEDNN) model and spatial attention maps based on difference images. Firstly, the difference image is generated by image processing. Then five binary images of difference images are obtained by using different thresholds. These binary images are regarded as spatial attention maps. We use the group convolution to reduce model complexity. Skip connection and 1 × 1 convolution are used to ensure good performance in the case of low depth. As an input, spatial attention map can be selectively fed into the input of each block. In addition, we only need to adjust the parameters of the classifier to easily extend to other datasets without

show abstract

“…Different measures also make it difficult to keep the results of emotion research consistent. However, methods proposed in the field of human recognition can be used more effectively ( 6 ) in psychological research and are suitable for real-time monitoring of the mental state of healthy people in a universal environment.…”

Section: Introductionmentioning

confidence: 99%

Mental states and personality based on real-time physical activity and facial expression recognition

Huang

Zhai

Song³

et al. 2023

Front. Psychiatry

View full text Add to dashboard Cite

IntroductionTo explore a quick and non-invasive way to measure individual psychological states, this study developed interview-based scales, and multi-modal information was collected from 172 participants.MethodsWe developed the Interview Psychological Symptom Inventory (IPSI) which eventually retained 53 items with nine main factors. All of them performed well in terms of reliability and validity. We used optimized convolutional neural networks and original detection algorithms for the recognition of individual facial expressions and physical activity based on Russell's circumplex model and the five factor model.ResultsWe found that there was a significant correlation between the developed scale and the participants' scores on each factor in the Symptom Checklist-90 (SCL-90) and Big Five Inventory (BFI-2) [r = (−0.257, 0.632), p < 0.01]. Among the multi-modal data, the arousal of facial expressions was significantly correlated with the interval of validity (p < 0.01), valence was significantly correlated with IPSI and SCL-90, and physical activity was significantly correlated with gender, age, and factors of the scales.DiscussionOur research demonstrates that mental health can be monitored and assessed remotely by collecting and analyzing multimodal data from individuals captured by digital tools.

show abstract

Action Units recognition based on Deep Spatial-Convolutional and Multi-label Residual network

Cited by 8 publications

References 14 publications

Deep neural network to extract high-level features and labels in multi-label classification problems

Deep neural network to extract high-level features and labels in multi-label classification problems

Computational efficient deep neural network with difference attention maps for facial action unit detection

Mental states and personality based on real-time physical activity and facial expression recognition

Contact Info

Product

Resources

About